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Preface 


The present volume contains hints or full solutions to many of the exer- 
cises in the first eight chapters of the text Probability: An Introduction, 2nd 
Edition, [@W14]| by Geoffrey Grimmett, Professor of Mathematical Statis- 
tics, University of Cambridge, and Dominic Welsh, Professor of Mathe- 
matics (retired), University of Oxford. Solving problems being an essential 
part of the learning process, my goal is to provide those learning and teach- 
ing probability with a large number of worked out exercises. Grimmet 
and Welsh’s textbook covers all the topics in probability that are usually 
taught at the undergraduate level for math students with calculus, but not 
measure theory: probabilities, random variables, discrete and continuous 
distributions, joint distributions, transformations, moments, conditioning, 
basic limit theorems, inequalities. Therefore this solutions manual can be 
helpful to anyone learning or teaching probability at the college level. 

As Grimmett and Welsh’s textbook appears simple until you try to do 
the exercises, I encourage the reader to work through all of the exercises. 
Many ideas are contained in a few words of typical probability problems 
such as dice, cards, coins, etc. To make the solutions concise, I have in- 
cluded only the necessary arguments; the reader may have to fill in the 
details to get complete proofs. 

Comments and questions on possibly erroneous solutions, as well as 
suggestions for more elegant or more complete solutions will be greatly 
appreciated. 


Huy Bui 
Georgia Tech, 2019 
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Part 1 


Basic Probability 


CHAPTER 1 


Events and probabilities 


1.1. Experiments with chance 
1.2. Outcomes and events 
EXERCISE (1.8). If A,B € F, show that AN Be F. 


SOLUTION. The complement of AN B equals (Q\A) U (Q\B), which is 
the union of events and is therefore an event. Hence AM B is an event, by 
(13), 

EXERCISE (1.9). The difference A\B of two subsets A and B of is the 
set AM (Q\B) of all points of 2 which are in A but not in B. Show that if 
A,BeéF, then A\Be Ff. 


SOLUTION. The complement of A\B equals (Q\A) U B, which is the 
union of events and is therefore an event. Hence A\B is an event, by (1.3). 


EXERCISE (1.10). The symmetric difference AAB of two subsets A and 
B of is defined to be the set of points of which are in either A or B but not 
both. If A, B € F, show that AAB € Ff. 


SOLUTION. AAB equals (A\B) U (B\A), which is the union of events 
(see Exercise 1.9) and is therefore an event. Hence AAB is an event, by 
(1.4). 

EXERCISE (1.11). If Ay, Ao,..., Am € F and k is positive integer, show 
that the set of points in which belong to exactly k of the A; belongs to F 
(the previous exercise is the case when m = 2 and k = 1). 

SOLUTION. First, we prove 

k 
() Ai, € F, 
j=l 
by induction on k, considering first the case k = 2 (obviously, the result if 
true for k = 1). Certainly the complement of A;, M Aj, equals ((\A;,) U 


11 
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(Q\A;,), which is the union of events and is therefore an event. Hence 
Aj, Aj, is an event, by (1.3). Thus the result is true for k = 2. Let h > 2 
and suppose that the result is true for k < h. Then 


h 
() Ai, € F, 
j=l 
so that 
h+1 h 
() Ai, = (1) Ai;) 7 Aings- 
j=l j=l 


Using the induction hypothesis, we obtain the result. Next, let A be the 
set of points in which belong to exactly k of the A; belongs to F. We have 
that 


k 
A U (() Ai,), 
(7) unions J=1 
which is the union of events and is therefore an event. Hence A is an event, 
by (1.4). 


EXERCISE (1.12). Show that, if 2 is a finite set, then F contains an 
even number of subsets of 2. 


SOLUTION. Define a mapping f: F > F by f(A) = 2\A for any event 
Aé¥F. The mapoing f is injective for if A; = Ao, then Q\A; = O\Ag, so 
f(A1) = f(A2). The mapping f is surjetive for if B € F, then f(Q\B) = B. 
Thus f is a bijective. Hence for any element A in ¥,there exists a unique 
Q\A # A, so F contains an even number of subsets of 2. 


1.3. Probabilities 


EXERCISE (1.17). Let pi, p2,...,pn be non-negative numbers such that 
pitpot::-+pn =1, and let ON = {w1,we,...,wn}, with F the power set 
of 9, as in Example 1.16. Show that the function Q given by 


Q(A) = ss Di for A € F, 
1:0j,EA 


is a probability measure on (Q, F). Is Q a probability measure on (Q, F) if 
F is not the power set of but merely some event space of subsets of 0? 
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SOLUTION. For the first part, it is not too difficult to check that a func- 
tion is a probability measure on (0, F) if and only if it satisfies conditions 
(a), (b) and (c) of Defintion 1.13. Condition (a): Q(A) > 0 for AE F 


because 


Q(A)= So p>0 for Ac F, 
t:0,EA 


since p; > 0 for all i. Condition (b): Q(Q) = 1 because 
N 
SOS y a= eee 
iwi EQ i=l 
Q(2) = 0 by its defintion. Condition (c): We first prove 
QL Ai) = 55 Q(Ai), 
i=1 i=1 


where Aj, Ag,..., An are disjoint events in F (in that A;7.A; = 9 whenever 
i #7), by induction on n, considering first the case n = 2. Then 


Q (Ai U Aa) 


Pi 
4:0;€A,UA2 
= S Pit DS Di 
tiw,E Ay 1:1w,€ Ag 


= Q(A1) + Q(A2). 


Hence the result is true for n = 2. Let m > 2 and suppose that the result 
is true for n < m. Then 
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14 
so that 
m+1 m 
QL Ai) = Q(LU Ai) U Amst 
i=l i=l 
= Q(LJ Ai) + Q(Amsi) since (LJ Ai) MN Amy = B 
i=1 i=l 
using the induction hypothesis 


= Ss" Q(Am) “Ty Q(Am-41) 


| 
© 
= 


and the induction step is complete. Also, since n is an arbitrary constant, 


it follows that 
QL Ai) = 5) OA), 
i=l i=1 


as desired. 
For the scond part, the answer to this question is yes. 


1.4. Probability spaces 
EXERCISE (1.19). If A, B € F’, show that P(A\B) = P(A) —P(ANMB). 


SOLUTION. The set A is union of the disjoint sets A\B and AN B, and 


hence 
by (1.14), 


P(A) = P(A\B) + P(AN B) 


giving that 
P(A\B) = P(A) —P(AN B). 


EXERCISE (1.20). If A, B,C € F, show that 
P(AUBUC) = P(A)+P(B)+P(C)—P(ANB)—P(BNC)—P(ANC)—P(ANBNC). 


1.4. PROBABILITY SPACES 15 


SOLUTION. We have (using the property that if A,B € F then P(AU 
B) = P(A) + P(B) — P(AN B)) that 
P(AUBUC) =P((AUB)UC) 
= P(AUB)+P(C) —P(AUB)NC) 
= P(A) + P(B) — P(ANB)+P(C)-P((ANC)U(BNC)) 
(A) + P(B) + P(AN B) +P(C) - [P(ANC)+P(BNC)-P(AN B)N(BNC))] 
(A) + P(B) + P(C) — P(AN B) —P(BNC) — P(ANC) — P(ANBNC). 
2 
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EXERCISE (1.21). Let A, B, C be three events such that 


5 7 6 
(4) ==, (B)= 5, ()= 5, 
P(ANB) == (BNO)=4, PAncy=2 
ol. 10’ ae iy 

1 

P(ANBNC)= i0° 


By drawing a Venn diagram or otherwise, find the probability that exactly 
two of the events A, B, C occur. 


SOLUTION. Let D be the event that exactly two of the events A, B, C 
occur. Then 
= [((ANB)\(ANBNC)U[L(ANC)\(AN BNC) U[(BNC)\(AN BNC). 
The events [(AN B)\(ANBNC)], [(ANC)\(AN BNC) and [((BNC)\(AN 
BO C)| are disjoint events, and so 
P(D) = P((ANB)\(ANBNC)) +P 
( 


(A 
= P(ANB)—-P(ANBNC)+P(A 
1 


ee ee ‘ 2 
10' 10 10 10° 10 





C)\\(AN BNC))+P(BNC)\(ANBNC)) 
C)—-P(ANBNC)+P(BNC)—-P(ANBNC) 
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EXERCISE (1.22). A fair coin is tossed 10 times (so that heads appears 
with probability $ at each toss). Describe the appropriate probability space 
in detail for the two cases when 

(a) the outcome of every toss is of interest, 

(b) only the total number of tails is of interest. 


In the first case your event space should have gz" events, but in the second 
case it need have only 2!! events. 


SOLUTION. (a) Representing heads by H and tails by T. The probability 
space of this experiment is the triple (0,7, P), where 
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(i) Q is the set of all ordered sequences of length 10 containing 
the letters H and T, where the kth entry of such a sequence represents the 
result of the kth toss, 

(ii) F to be the set of all subsets of Q, 

(iii) each point w in Q has equal probability, so that 


1 
P(w) = gio” 


and, more generally, 
P(A) = i |A| for each AC F. 


for A € F¥. 
(b) The probability space of this experiment is the triple (0,7,P), 

where 

(i) Q = {i: i =0,1,...,10}, the set of all integers between 0 and 
10, 

(ii) F to be the set of all subsets of Q, 

(iii) for each i € Q, we define the probability that 7 is the actual 
outcome by 


(") 
Pi) = Sra fori =0,1,...,10, 
and, more generally, 
PA) = S > P(i) for each AC F. 
iE A 


for Ac fF. 


1.5. Discrete sample spaces 


EXERCISE (1.25). Show that if a coin is tossed n times, then there are 


exactly 
n n! 
(;) ~ ki(n—k)! 


sequences of possible outcomes in which exactly & heads are obtained. If 
the coin is fair (so heads an tails are equally likely on each toss), show that 
the probability of getting at least k heads is 


=> (") 
Qn r)° 
r=k 
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SOLUTION. For the first part, representing heads by H and tails by T, 
the sample space is the set of all ordered sequences of length n containing 
the letters H and T, where the ith entry of such a sequence represents the 
result of the [th toss. When we choose & entries in order from n entries, 
the first entry can be chosen in n ways, leaving n — 1 choices for the second 
entry, and so on. After k — 1 entries have been chosen there remain n — 
(k —1) -n—k+1 entries from which to make the kth choice. Hence, the 
total number of ways of making these choices is n(n — 1)---(n —k +1). 
This number can be rewritten as 





n(n—1)---(n—k+1)(n—k)(n—k-1)---2-1 
(n—k)(n—k-1)---2-1 ; 





and using the factorial notation, we can write this expression succinctly as 


n! 
(n—k)! 


However there are k! different sequences that have exactly k heads. Thus 
capt gives the number of sequences having exactly & heads, when each 
sequence is counted k! times. Hence the number of sequences of possible 


outcomes in which exactly & heads are obtained is 


Gee 


For the last part, the sample space is the set of possible outcomes. It 
has 2” elements, each of which is equally likely. Let A be the event that 
getting at least k heads. Then 





P(A) = 5° P(ur). 
r=k 


where w,, for r= k,k+1,...,n, is the event of getting exactly 7 heads. By 


the above, 
1 fn 
P LNs mee ; 
(wi) = 5 (*) 


giving that 
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EXERCISE (1.26). We distribute r distinguishable balls into n cells at 
random, multiple occupancy being permitted. Show that 

(a) there are n” possible arrangements, 

(b) there are (;)(n—1)’~* arrangements in which the first cell contains 
exactly k balls, 

(c) the probability that the first cell contains exactly k balls is 


() (G) 0-3) 


SOLUTION. (a) If there are r successive choices to be made in order to 
put r distinguishable balls into n cells at random and, for 1 <i <r, the ith 
choice can be made in n ways since multiple occupancy being permitted, 
the the total number of ways of making these choices is n”. 

(b) There are (2 ways to arrange r balls into the first cell. The number 
of ways to arrange r — k balls into n — 1 remain cells is (n — 1)"~* by part 
(a). Thus there are (7) (n—1)"~* possible arrangements which the first cell 
containing & balls. 

(c) The sample space is the set of possible outcomes. It has n” elements, 
each of which is equally likely. Let A be the event that the first cell contains 
exactly & balls. Then A has (7) (n — 1)’-* elements. The probability of A 


Ooo" -(@) 2)" 
GG.) - 


EXERCISE (1.27). In a game of bridge, the 52 cards of a conventional 
pack are distributed at random between the four players in such a way that 
each player receives 13 cards. Show that the probability that each player 
receives one ace is 





94° ARIS 137 


=0.105.... 
Bal 0.105 
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SOLUTION. The sample space is the set of possible outcomes. It has 


(13) (is) (i) 


elements, each of which is equally likely. There are 


nm 48\ (36\ (24 
SG bi ye ie 
ways that each player receives one ace. Therefore, the answer is 


(1) (13) (72) _ 24-48! - 134 


(is) (is)(is) 52! 
= 0.105... 





EXERCISE (1.28). Show that the probability that two given hands in 
bridge contain & aces between them is 


(:) (25 =1)/(ee) 


SOLUTION. The sample space is the set of possible outcomes. It has 
Gs) elements, each of which is equally likely. There are (;) as 2 ways that 
one player receives k ace. Therefore, the answer is 


Hees 
@) 


EXERCISE (1.29). Show that the probability that a hand in bridge con- 
tains 6 spades, 3 hearts, 2 diamonds and 2 clubs is 


13\ (13\ (13\7 /(2 
6 3 2 13]° 
SOLUTION. The sample space is the set of possible outcomes. It has 
(=) elements, each of which is equally likely. There are Ce) es (yr ways 
that a hand in bridge contains 6 spades, 3 hearts, 2 diamonds and 2 clubs. 


Therefore, the answer is 
13\ (13\ (13\? / 52 
6 3 2 13)° 
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EXERCISE (1.30). Which of the following is more probable: 
(a) getting at least one six with 4 throws of a die, 
(b) getting at least one double six with 24 throws of two dice? 


This is sometimes called ’de Méré’s paradox’, after the professional 
gambler Chevalier de Méré, who believed these two events to have equal 
probability. 


SOLUTION. The probability of getting at least one six with 4 throws of 


a die is 
5\4 
1—{—) =0.5177--- 
(3) : 


which is slightly higher than the probability of getting at least one double 
six with 24 throws of two dice, 


1.6. Conditional probabilities 


EXERCISE (1.34). If (Q,4,P) is a probability space and A, B, C are 
events, show that 


P(ANBNC)=P(A| BNC)P(B | C)P(C) 
so long as P(B(\C) > 0. 
SOLUTION. We have that 
P(ANBNC)=P(AN(BNC)) 
=P(A|(BNC))P(BNC) by (1.32) 
= P(A | (BN C))P(B | C)P(C) 
so long as P(B(\C) > 0. 





























EXERCISE (1.35). Show that 





P(B| A) =P(A| B) 














if P(A) > 0 and P(B) > 0. 
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SOLUTION. We have that 

P(BM A) 
P(A) 

P(AN B) 
P(A) 

P(A | B)P(B) 


eet Wd by (1.32) 





P(B| A) = by (1.32) 
































=P(A| B) 














if P(A) > 0 and P(B) > 0. 


EXERCISE (1.36). Consider the experiment of tossing a fair coin 7 times. 
Find the probability of getting a prime number of heads given that heads 
occurs on at least 6 of the tosses. 


SOLUTION. The probability space of this experiement is the triple (Q, F,P), 
where 

(i) Q={H;:0<1< 7}, (A; is the event that heads occur i times) 

(ii) F is the set of all subsets of Q, 

(iii) each point in 2 has 


Let A be the event that we get 2, 3, 5, or 7 heads and B be the event that 
heads occur 6 or 7 times. We neet to find P(A | B). The events A and B 
are subsets of Q given by 

A = {Ao, fiz, Hs, H7}, 

B= {Hg, Hy}. 


The event B has 
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Finally, AM B is given by 


ANB = {Hr}, 
so that 
P(AN B) = (3) 
and 
P(A| B) = ae ) 
G)" 





1.7. Independent events 
EXERCISE (1.42). Let A and B be events satisfying P(A),P(B) > 0, 
and such that P(A | B) = P(A). Show that P(B | A) = P(B). 
SOLUTION. We have that 
P(BM A) 





P(B| A) = by (1.82) 











P(ANM B) 




















eae eee ae by (1.32) 



































since P(A | B) = P(A) 














as required. 


EXERCISE (1.43). If A and B are events which are disjoint and inde- 
pendent, what can be said about the probabilities of A and B? 


SOLUTION. By the definition of independence events, 
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P(A)P(B) = P(AN B) 
=0 
since A and B are disjoint. Thus, either A or B must have zero probability. 


EXERCISE (1.44). Show that events A and B are independent if and 
only if A and Q\B are independent. 


SOLUTION. Suppose that A and B are independent. Writing P(AN 
(Q\B)) = P(A\B) (see Exercise 1.9). The set A is union of the disjoint 
sets A\B and AN B, and hence 


























P(A) = P(A\B) + P(ANB) by (1.14), 
giving that 
P(A\B) = P(A) — P(An B) 
= P(A) — P(A)P(B) since A and B are independent 
= P(A). — P(B)) 
= P(A)P(Q\B) 


Thus 


P(AN (Q\B)) = P(A)P(Q\B), 


so A and (\B are independent. 
Conversely, suppose that A and (\B are independent. By the above, 
A and B = Q\(Q\B) are independent 


EXERCISE (1.46). If Ay, Ag,..., Am are independent and P(A;) = p for 
7=1,2,...,m, find the probability that 

(a) none of the A; occur, 

(b) an even number of the A; occur. 


SOLUTION. (a) Since Aj, Ag,..., Am are independent, 0\Ai1,2\A2,...,Q\Am 


are also independent (see Exercise 1.45). Thus 


P((Q\A1) 9 (O\A2) N+ (Q\Am)) = PQ\AL)P(O\ Ag) ---P(Q\ Am) 
= (1—p)”. 
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(b) Let B be the event that there is an even number of the A; occur. 
Then 


_ MM \ 26 mi ie 2i+1_m—2i-1 (4 
, (i) 4 > (6,74)? 3 » @) 
<2i<m 


0<2i+1<m 


I 


(q+p)” 


™M\ 94 m—2i m 2i+1 m—2i-1 2 
oS (i)? as De aa? : eA) 
0<21<m 


0<2i+1<m 


Adding equations (1) and (2) to obtain 


m . . 
1 —p)"=2 24 ,m—2i 
T=) S (Sr) q 


0<2i<m 
= 2P(B), 
giving that 


where p+q=1. 


EXERCISE (1.47). On your desk, there is a very special die with a prime 
number p of faces, and you throw this die once. Show that no two events A 
and B can be independent unless either A or B is the whole sample space 
or the empty set. 


SOLUTION. We may take 2 = {1,2,...,p}. Consider two events A and 
B. If ANB=9, A#@ and BQ, then P(A)P(B) > 0 = P(AN B), so 
A and B are dependent. If AN B # @ and A and B are independent, then 
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ANB A| |B 
pang) — 408) _ |4l 1B) 
Pp Pp Pp 





implying 


|AN B] = az al, 

Pp 
Since |A/NM B| is an integer, it follows that p | |A||B|, giving that p | |A| or 
p | |B|. This implies that |A| = p or |B| = p since |A|,|B| < p. Hence, 
A=QorB=90. If A= @ or B= @, then the result is trivial. Therefore 
no two events A and B can be independent unless either A or B is the 
whole sample space or the empty set. 


1.8. The partition theorem 


EXERCISE (1.52). Here are two routine problems about balls in urns. 
You are presented with two urns. Urn I contains 3 white and 4 black balls, 
and Urn II contains 2 white and 6 black balls. 

(a) You pick a ball randomly from Urn I and place it in Urn II. Next 
you pick a ball randomly from Urn II. What is the probability that the ball 
is black? 

(b) This time, you pick an urn at random, each of the two urns being 
picked with probability 7 and you pick a ball at random from the chosen 
urn. Given the ball is black, what is the probability you picked Urn I? 


SOLUTION. Let A be the event that I pick a white ball from Urn I and 
place it in Urn I] and B be the event that I pick a black ball from Urn II. 
The pair A, A° is a partition of the sample space (since exactly one of them 
must occur) 


EXERCISE (1.53). A biased coin shows heads with probability p = 1—q 
whenever it is tossed. Let u, be the probability that, in n tosses, no two 
heads occur successively. Show that, for n > 1, 


Unt2 = QUn+1 + Pqun, 
and find un by the usual method (described in Appendix B) when p = 3. 


SOLUTION. For the first part, let A be the event be the event that no 
two heads occur successively in n + 2 tosses and B be the event that the 
final toss shows tails. The pair B, B® is a partition of the sample space 
(since exactly one of them must occur). Then 
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P(A | B°) = pqun, 


and 


PCA: B) = guns 
By Theorem 1.48, 


P(B) = P(A)P(B | A) + P(A°)P(B | AY), 
which is to say that 


Un+2 = QUn+1 + pqun, 


as required. 
For the last part, we shall find the solution of the difference equation 


Un+2 — QUn+1 — Pqun = 0 


subject to the boundary conditions wu, = 1, u2 = 1— 3 3 = 2. The auxiliary 
equation is 

1 2 

@? —--@-—--=0 

3 9 
with roots 0 = 3, —5. The general solution is therefore 

2 1 

Un = a(3)" oa (—3)", 

where the constants a, b are found from the boundary conditions to be 
given by a = 4, b= —i. 


1.9. Probability measures are continuous 
1.10. Worked problems 
1.11. Problems 


PROBLEM (1.11.1). A fair die is thrown n times. Show that the prob- 
ability that there are an even number of sixes is 5 [1 + (2)"]. For the 
purpose of this question, 0 is an even number. 
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SOLUTION. We prove this by induction on n, considering first the case 
n = 0. Certainly the probability that there are an even number of sixes is 
1 which is agreement with 5 1 + 3)"; roughly speaking, this is because 
since if the fair die is thrown 0 times, then there is 0 (an even number) 
sixes. Hence the result is true for n = 0. Let m > 0 and suppose that the 
result is true for n < m. Then consider the case that a fair die is thrown 
m+ 1 times. Let A be the event that there are an even number of sixes 
among m+ 1 throws and B be the event that there are an even number of 
sixes among the first m throws. The pair B, B° is a partition of the sample 
space (since exactly one of them must occur). It is easy to see that 


P(A| B) => 


since if B occurs, then A occurs if and only if the result of the (n + 1)th 
throw is a number other than six. Similarly, 


é 1 
P(A | B®) ==. 


By Theorem 1.48, 


P(A) =P(A| B)P(B) + P(A | B°)P(B*) 





es Sus a: 
=7-5il (SY) + 5+ {1 511+ (5)"]} by the induction hypothesis 
ee: 
Set “ym+1l 
s+ Gye 


The result follows by induction. 


PROBLEM (1.11.2). Does there exist an event space containing just six 
events? 


SOLUTION. The answer to this question is no. We prove this by contra- 
diction. Assume there exists an event space F containing just six events. 
The event space * must contain the empty set @ and the whole set 2. 
Since the event space F has six events, it follows that {@,0, A, A°, B, B°} 
where A,B £0 with A Band B # A®. Since A,B € F, s0 ANB = 
(ASU BY) € F by (1.4). IF ANB =Q, then A = 9 and B = Q, which 
contradicts the fact that A,B 4 Q. If AN B = B, then B C A, and so 
A\B = AN Bo e€F. This is a contradiction since A\B 4 A, A\B & B, 
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A\B 42, A\B#Q, A\B # A, A\B & B®. Similarly, AN B = A will 


yeild a contradiction. Thus 


ANB=@. (1) 


Since A, B € F, it follows that AUB € F. If AUB #Q, then 0\(AUB) = 
(AU B)* € F. But (AUB)* #4 A, (AUB) F B, (AUB) £ AS, 
(AUB)*° # BS, (AUB)* # @, (AUB)* 4 Q, so (AUB)* ¢ F, a contradiction. 
Thus 


AUB=9. (2) 


From (1) and (2) we obtain that B = A°, which contradicts the fact that 
B# A°. Therefore there does not exist an event space containing just six 
events. 


PROBLEM (1.11.3). Prove Boole’s Inequality: 


+(a) <r 


SOLUTION (1). Let By = Ay and B; = A;\(Ui_) Ax) for i = 2,3,. 
Then 


a P(B;). 
i=1 
However, by construction B; C A; for i= 1,2,...,n, so 
P(Bi) < P(A;) Lope Lane ot 


and so 
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+(Ga) <seun 
4=1 4=1 


as required. 
Note that this proof requires to show that U;_, B; is the union of dis- 
joint events in F such that 


UB=Ua4 
i=1 i=1 
where 
= Aj 
i-1 
B; = Ai\((J Ae) for 4 = 2,3,..., 
jewel 


We first show that B.A B, = 0 for r 4 s. Without loss of generality, 
assume that r < s, then for any « € Bs, x € AN, A; so that x € As 


and x ¢ We A;. Thus x ¢ A, and hence x ¢ B,. We next show that 


n 
Indeed, for any « € Uj, Bi then x € - for some h € {1,2,...n}, so that 
x € Ay\ Usa) Aj. Thus  € Ap C UL, Ai. So 


U B,C U Aj. (1) 


Conversely, for any « € Uj, Ai, then x € Ax for some k € {1,2,...,n}. 
If x € Ap and x € Usa) A a Treat Soa éc. Ifx € Ay and 
r¢ Wey | Aj, then x € Ar\ Usa | Aj = Br. So a € US, Bi. Hence 


tC 


U4rc UB: (2) 
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i=1 i=1 


as required. 


SOLUTION (2). The first inequality is trivially true ifn = 1. Let m > 1 
and assume that the inequality holds for n < m. Then 








m+1 m m 
P(() As) = P(LJ As) + P(Am41) — P(LJ(Ai 9 Am4i)) 
i=1 i=1 1 


























IA 
J 
ee 


Aj) + P(Am+1) < } P(A,), 


ll 
an 
a 
ll 
mn 


by the hypothesis. The result follows by induction. 
PROBLEM (1.11.4). Prove that 


i=1 i=1 


This is sometimes called Bonferroni’s inequality, but the term is not rec- 
ommended since it has multiple uses. 


SOLUTION. We have that 
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P(\A) =PQ(\(4N) using (48° 


n 
= PU A§)°) using De Morgan’s Laws 


n n 
>1- S° P(A®) using Boole’s inequality P(LJ 
i=1 


Bi) < 5 PB) 
i=1 


as required. 


PROBLEM (1.11.5). Two fair dice are thrown. Let A be the event that 
the first shows an odd number, B be the event that the second shows an 
even number, and C be the event that either both are odd or both are even. 
Show that A, B, C are pairwise independent but not independent. 


SOLUTION. The probability space of this experiment is the triple (Q, F, P), 
where 

(i) Q= {(4,7) 4,7 =1,2,...,6}, the set of all ordered pairs of integers 
between 1 and 6, 

(ii) F is the set of all subsets of Q, 

(iii) each point in Q has equal probability, so that 


1 
P((i,9)) = 3 for i,j =1,2,...,6, 


and, more generally, 


P(s)= (SI for each S CQ. 
The events A,B and C are subset of 2 given by 
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A = {(i,j):i=1,3,5 and j =1,2,...,6}, 
B = {(i,j):i=1,2,...,6 and j =2,4,6}, 
C = {(i,j):i and J are either both are odd or both are even}. 


The event A contains 3 x 6 = 18 ordered pairs, B contains 6 x 3 = 18 
ordered pairs, and C contains 3 x 3+3 x 3 = 18 ordered pairs, giving that 


Finally, 


ANB =({(i,j):i=1,3,5 and j = 2,4, 6}, 

AD Ga t(j} sta 153; oan pS Boats 

BOG = {i )ct=2.4 band 7 = 254,615 
ANBNC=6. 


The event AM B contains 3 x 3 = 9 ordered pairs, ANC contains 3 x 3 = 9 
ordered pairs, BMC contains 3 x 3 = 9 ordered pairs, and AN BOC 
contains no ordered pair, so that 

































































P(ANB) = = = = 55 = P(A)P(A), 

(ANC) = =3=5-5=PUPO), 

P(BNC) == =7=5:5=PBPO), 
P(AN BNC) =04 == 5-5-5 =P(A)P(B)P(C). 


Therefore the events A, B, C are pairwise independent but not independent. 


PROBLEM (1.11.6). Urn I contains 4 white and 3 black balls, and Urn 
II contains 3 white and 7 black balls. An urn is selected at random, and 
a ball is picked from it. What is the probability that this ball is black? If 
this ball is white, what is the probability that Urn I was selected? 


SOLUTION. Let A be the event that a black ball is picked and let B be 
the event that the first urn is selected. The pair B, B°is a partition of the 
sample space (since exactly one of them must occur). We have 
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3 it 
P(A|B)==,P(A|B)=—=— 
(4| B)==,P(A| B) ==, 

1 if 

BiB) =P (BS _. 
(B) = 5,P(B) =5 


By Partition theorem, Theorem 1.48, 


P(A) = P(A|B)P(B)+P(A| B)P(B) 
Bd ee a 
7.2 102 
_ 79 
- =. 


We shall calculate P(B|A‘°). We have P(A°) = 1-— P(A) = 1- 
79/140 = 61/140. By Bayes’s theorem, Theorem 1.50, 


ae B® 





P(A‘) 
4 1 
ce etip; OL, 
61 a 
Tio 490 
[oxe) COO0e@e 
C0o0@ee@e eeeee 
Urn I Urn II 





PROBLEM (1.11.7). A single card is removed at random from a deck of 
52 cards. From the remainder we draw two cards at random and find that 


they are both spades. What is the probability that the first card removed 
was also a spade? 


SOLUTION. Let A be the event that the first card removed was a spade 
and B be the event that from the remainder we draw two cards at random 
and find that they are both spades. The pair A, A® is a partition of the 
sample space (since exactly one of them must occur). It is easy to see that 
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12 
P(B| A) = aT 
P(B| A‘) = a 
P(A) = 52) 
P(A‘) = 59° 
By Partition theorem, Theorem 1.48, 
P(B) = P(B|A)P(A)+P(B|A‘)P(AS) 


12 Big ae 39 
dl 52 51 52 





We shall calculate P(A 


P(A|B) 


PROBLEM (1.11.8). A fair coin is tossed 3n times. Find the probability 
that the number of heads is twice the number of tails. Expand your answer 
using Stirling’s formula. 


SOLUTION. The sample space is the set of possible outcomes. It has 
23” elements, each of which is equally likely. There are a) ways to throw 


so that that the number of heads is twice the number of tails. Therefore, 


the answer is 
1 /3n 
gan ba (1) 


To understand how this behaves for large n, we need to expand the 
binomial coefficient in terms of polynomials and exponentials. The relevant 
asymptotic formula is called Stirling’s formula, 
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n! ~ (n/e)"V2an as 1 —> 00, (2) 


where ay, ~ bp, means a,,/bp — 1 as n + oo. See Theorem A.4 for a partial 
proof of this. 
Applying Stirling’s formula to (1), we obtain 


1 /3n 3n)! 
am (>, = on 
yin__(8n/3) "2A 
(2n/e)2”/272n(n/e)”V/2rn 
g-3n (3n)*" V3 
(2n)??n® 2/7 
ge80 aos 
2202 /n 


(n) am 


The factorials and exponentials are gigantic but they cancel out. 











PROBLEM (1.11.9). Two people toss a fair coin n times each. Show that 
the probability they throw equal numbers of heads is 


eee 


SOLUTION. Firstly, we shall show that 


Indeed, 


2n 2n 2n 2n 
+a) =1+ xr Lope Mas ec x 
(1 ee 1 2 n 2n 
1 2 n 2n 


and 
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1+ (tle ( 
(1)+()}e* 1G) +) +( 
Cr) + G2 )G) +622) G)+-+6) 


Thus the coefficient of "in the expansion of (1+2)?" is (2); The coefficient 
of x” in the expansion of (1+ x)"(14+ 2)” is 


OG) 


since (7) = (,,",). Thus 


(3) +) ++ 0) =@): 


There are C) where k € {0,1,...n} ways for first person or the second 

















NW, wd 





+ 








n 


person toss k heads. So there are ( Ve ways for both people have the 
number of heads equal k. The number of cases both people throw equal 
numbers of heads is 


i) =) 8) PG) Ge). 


Thus the probability they throw equal numbers of heads is 


an\ (1\7" 
mp N2P > 
PROBLEM (1.11.10). In the circuits in Figure 1.2, each switch is closed 


with probability p, independently of all other switches. For each circuit, 
find the probability that a flow of current is possible between A and B. 


SOLUTION. For the first circuit in Figure 1.2, let C’ be the event that a 
flow of current is impossible between A and B, and D, E, F' be the events 
that the middle, top, bottom paths are discnnected, respectively. Then 


lI 
aN 
SS 
Na 
i) 
+ 
a ™~ 
pe 3 
Noe 
iw) 
+ 
+ 
i 
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P(C) = P(D)P(E)P(F). 
Note that P(D) = 1— p, and P(E) = P(F) = (1 — p”). Thus 


P(C) = (1—p)(1—p*)’. 


Hence 


P(C°) =1-—P(C) 
=1-(1-p)(1-p’). 
For the second circuit in Figure 1.2, let C’ be the event that a flow of 
current is impossible between A and B, and D be the event that the middle 


vertical switch is open. The pair D, D° is a partition of the sample space 
(since exactly one of them must occur). By Theorem 1.48, 


P(C) = P(C | D)P(D) +P(C | D)P(D°). 


Note that P(C | D)P(D) = (1 — p)(1 — p?). In order to calculate P(C | 
D)P(D‘°), notice that impossble current flows given that the middle vertical 
switch is closed if and only if the two switches on the left are both open or 
the the two switches on the right are both open. This equivant to the fact 
that possible currents flows if and only if the two swiches on the left are 
not both open and the two switches on the right are not both open; this 
situation happens with probability [1 — (1 —p)?]?, hence P(C | D)P(D°) = 
p{1—(L—(1—p)?]}2. Thus P(C) = (1—p)(1—p?) +p{1 — [1 - (1 —p)°]P. 
Therefore, 





P(C°) =1-P(C) 
=1-(1—p)(1—p’) +p{1 — [1 — (1 p)7J}’. 


PROBLEM (1.11.11). Show that if un is the probability that n tosses of 
a fair coin contain no run of 4 heads, then for n > 4 


1 1 1 1 
Un = gvn-l + quin-2 g in-3 16 "4 
208 


Use this difference equation to show that ug = 555. 
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SOLUTION. For the first part, let A be the event that n tosses of a fair 
coin contain no run of 4 heads, B;, 7 = 1, 2,3, 4 be the event that n—7 tosses 
of a fair coin contain no run of 4 heads. {B,, Bo, B3, B4} is a partition of 
the sample space (since exactly one of them must occur). It is easy to see 
that 


























1 
P(A| Bi) = 5 
1 
P(A | Ba) = 7 
1 
P(A Bs) = 8 
x 1 
(A| Ba) = 75 
By Theorem 1.48, 
P(A) = P(A| B,)P(Bi), for i = 1,2,3,4 
which is to say that 
= et oa ea 
Un = gvn-l quin-2 t g in—3 t 16" 


as required. 
For the last part, note that uy = 1, ug = 1, u3 = 1, ua (3)4 
Thus 





al 
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it iA if 1 
Us = —U4 + —ug + =U 4 Uw 
5 94 aie 2 ie: 
iy Shae al 1 1 15 
=_. 1 14 -l= 
oie a ot 16 29 
apie aby yi yl 
U6 = 9 ts T Fics Fr >U3 4 16° 
_1 1 1 ee ee pee 
“9 39° 2°46 8 ~ 16° 16 
Be Sendhil 
U7 = 5 6 t he T gtd T 16 3 
Lt Cis ted 1d ,_B 
~2 146°4 32'8 16°16 °°» 32 
aS adios ofa ol 
Us = gl7 T qué t ee? T 16/4 
Tee a A A He; 
2 32'4 16 8 32 16 16 8 


PROBLEM (*1.11.12). Any number w € [0, 1] has a decimal expansion 
w=0.41%2..., 


and we write f;,(w,n) for the proportion of times that the integer k appears 
in the first n digits in this expansion. We call w a normal number if 


fr (w,n) 


> — as N > CO 

10 
for k = 0,1,2,...,9. On intuitive grounds we may expect that most num- 
bers w € [0,1] are normal numbers, and Borel proved that this is indeed 
true. It is quite another matter to exhibit specific normal numbers. Prove 
the number 


0.1234567891011121314... 


is normal. It is an unsolved problem of mathematics to show that e—2 and 
am — 3 are normal numbers also. 


PROBLEM (1.11.14). (a) Let P(A) denote the probability of the occur- 
rence of an event A. Prove carefully, for events A), A2,...,An, that 
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n 


PU Aj) = DPA) —S°P(A;N Aj) + S- P(A; N Aj N Ag) ay 


i=1 1<J i<j<k 





+(-1y"4P() A). 


(b) One evening, a bemused lodge-porter tried to hang n keys on their 
n hooks, but only managed to hang them independently and at random. 
There was no limit to the number of keys which could be hung on any hook. 
Otherwise, or by using (a), find an expression for the probability that at 
least one key was hung on its own hook. 

The following morning, the porter was rebuked by the Bursar, so that in 
the evening she was careful to hang only one key on each hook. But she 
still only managed to hang them independently and at random. Find an 
expression for the probability that no key was then hung on its own hook. 
Find the limits of both expressions as n tends to infinity. 

You may assume that, for real a, 


(Oxford 1978M) 


SOLUTION. (a) We prove this by induction on n, considering first the 
case n = 2. Certainly, Ay U Az = A; U (A2\A1) is a disjoint union, so that 








P(A U Ag) = P(A1) + P(A2\A1) 
P(A1) + P(Aa\(A1 9 Aa) 
= P(A1) + P(A2) — P(A, N AQ). 





























Hence the result is true for n = 2. Let m > 2 and suppose that the result 
is true for n < m. Then it is true for m events. That is to say 


°(Ua) = P(A) — P(A Ay) + S> P(Ain Aj Ax) — +++ + (-1)"'P (A«), 


i<j t<j<k 


so that 
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P (Ai Aj M Ax) — 


i<j<k<m 


sarin (fs) ortnnr(Gs}oa) 























= )>P(Ai) + P(Am+1) —P (U4 .) Ome} am P (Ai Aj) 
+ SD P(Ain Ayn Ai) - cme (Fa) 
m+1 m 
= Frcay-P(Ytaintnen) — D(A: n Ay) 


i<j<k<m 


see 


t<j<k<m 


— S> P(Ain Aj) + 


i<j<cm 





P (Ag Aj Ag) — 0 + (—1)™ BP (A A; 


$5 P(AiN Aj Am+41) 


i<jcm 


i=l 


S 7 P (Ai M1 Am+1) — 
i=1 


m+1 














P(A; Aj N Ag NM Amoi) — +++ + (-1)™*'P ( 


i<j<m41 


pies oh ag ( 


() A: 


i=l 


yo 


(oes Base (A 4) 


i=1 


So P(AiN Aj Ax) — 


i<j<k<m 


So P(Ain Aj) + 


| 


oz 


i<j<k<m41 


P(A;N Aj N Ak) — 


m+1 


() Ai 


i=l 


The result follows by induction. 


(b) Consider n keys 


that the ith key hung on its own hook, where i € {1,2,... 


was hung on their n hooks. Let A; be the event 
,n}. Then 
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n—-1)! 1 
P(A,) = | as 
nN: nm 
— 2)! 
P(A Ay) =o uy for i<j 
—k)! 
P(Aa, A M+ Ai) = OO for i1 < ig <-++ < ty 


Let A be the event that ther is at least one key hung on its own hook. Then 


Pa) =P(U} 


= 50> (Ai) — 55 P(AiN Aj) + $5 P(AiN Aj Ax) — + 


tJ t<j<k 


+ (1) Pe (A 4.) 


1 n\ 1 n\ 1 naifn)\ 1 
mer (sla (3)a meee yen (n) a 
1 


We have that 

















2 3 
vc __ ca as eee 
See ga igs : 
and so 
= ieee ee) 1 1 1 Th, 
“= Fol Spal Bt 
Thus 


a a | as n> ©. 
! ! e 


Therefore, 
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P(A) =1-- as 1 —> 00, 


e€ 
1 1 
P(A‘) =1-—P(A) =1-(1--)=- as 2 — OO. 
e€ e€ 
PROBLEM (1.11.15). Two identical decks of cards, each containing N 
cards, are shuffled randomly. We say that a k-matching occurs if the two 
decks agree in exactly k places. Show that the probability that there is a 


k-matching is 





aga aT a 
for k = 0,1,2,...,.N. We note that a, ~ 1/(k!e) for large N and fixed k. 
Such matching probabilities are used in testing departures from randomness 


in circumstances such as psychological tests and wine-tasting competitions. 
(The convention is that 0! = 1.) 


SOLUTION. Let A be the event that a k-mathing occur and B be the 
event that 0-matching occur in N — k remaining cards. Since A and B are 
independent, so 























Th= P(ANM B) 
= P(A)P(B) 
= (are 
N\(N—k)! 1 1 = 1 
=(7); Mi f Te as a ee 
ee eee (-1)"-* 
ae eo 3h ae 


as required. We note that a, ~ 1/(k!e) for large N and fixed k. 


PROBLEM (1.11.16). The buses which stop at the end of my road 
do not keep to the timetable. They should run every quarter hour, at 
08.30, 08.45, 09.00,..., but in fact each bus is either five minutes early or 
five minutes late, the two possibilities being equally probable and different 
buses being independent. Other people arrive at the stop in such a way 
that, t minutes after the departure of one bus, the probability that no one 
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is waiting for the next one is e~'/>. What is the probability that no one 
is waiting at 09.00? One day, I come to the stop at 09.00 and find no one 
there; show that the chances are more than four to one that I have missed 
the nine o’clock bus. 

You may use an approximation e® ~ 20. (Oxford 1977M) 


SOLUTION. For the first part, let A be the event that no one is waiting 
is waiting at 09.00, B, be the event that the last bus depart at 08.55, Bo 
be the event that the last bus depart at 08.50 and Bs be the event that the 
last bus depart at 08.40. {B), Bz, B3} is a partition of the sample space 
(since exactly one of them must occur). It is easy to see that 








P(A;) = _ fori =1,2,3 
P(A | Bi) =e", 
P(A | Ba) = eh +e, 
P(A | B3) = e+. 








P(A) = P(Bi)P(A | Bi) + P(B2)P(A | Bo) + P(B3)P(A | Bs) 


1 1 1 
Ses rian +e77) + a 





For the last part, by Bayes’ theorem, Theorem 1.50, 


P(A | B,)P(B1) 





P(B, | A) = 





P(A) 
¢(2e7! + e~? + e~4) 
23 
= 0.827. 
2e2 +e2 +1 — 


PROBLEM (1.11.17). A coin is tossed repeatedly; on each toss a head 
is shown with probability p, or a tail with probability 1— p. The outcomes 
of the tosses are independent. Let FE denote the event that the first run of 
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r successive heads occurs earlier that the first run of s successive tails. Let 
A denote the outcome of the first toss. Show that 


P(E | A= head) =p’! + (1 — p""!)P(E | A = tail). 


Find a similar expression for P(E | A = tail), and hence find P(E). (Oxford 
1981M) 


SOLUTION. For the first part, suppose r = s = 1. We have that P(E | 
A = head) = 1 and 
p14 (1—p" 1)P(E | A= tail) = p? +0-P(E | A= tail) = 1. 


Thus 


P(E | A= head) = p" !+4 (1—p"1)P(E | A= tail). 
Suppose r = 2, s = 1. We have that P(E | A = head) = p? and 
p+ +(1—p"))P(E | A = tail) = p' + (1—p*)-0=p. 


Thus 


P(E | A=head) 4 p" !4 (1—p" 1)P(E | A= tail). 


Suppose r = s. Let B be the event that the first run of r successive 
heads occurs in r first tosses (HH ...HT...). Let C be the event that the 


r times 
first run of r successive heads occurs earlier that the first run of r successive 


tails and A = head. Thus EM (A = head) = BUC, BNC = 9, and so 


P(EN(A=head)) = P(B)+P(C) 
p’ + P(A = head)P(B’)P(E | A = tail) 
= p’+p(1—p"|)P(E | A= tail) 





where B’ be the event that there are at least the tail occurs from second 
toss to rth toss. Thus 
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Bee es ety eee ee) 











P(A = head) 
p’ +p(1 —p"*)P(E | A = tail) 
Pp 


=p" 14 (1—p"1)P(E | A= tail). 

For the second part, let M be the event that r tails occur in r first 
tosses. Let N be the event that there are at least one head occur from the 
second toss to the rth toss. Then 

E | (A= tail) = (A = tail) A NO (E | A = head). 
Since (A = tail), N, NM (E| A = head) are independent events, so 


P(E | A=tail) =(1—p)[1-(1- p)”"] P(E | A= head). 


Let « = P(E | A= head) and y = P(E’ | A = tail). From the above result, 
we hvae 














c=p"'+(1—p"")(1—p)[1-(1-p)""] 2 
= p+ ={1-(1-p)[1-(1—p)"""}}a 
po _ grt 
-_ *1-(G-pil-G-py] G-py tp 
7 ray Po tp) +p] 





We have 
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P(E) = P(A = head)P(E | A = head) + P(A = tail)P(E | A = tail) 








eee a: p [A —p)’+p-]] 
Pap) +p GP) (lp) ep 
pp =p) ep 





Cpt 


PROBLEM (*1.11.18). Show that the axiom that P is countably additive 
is equivalent to the axiom that P is finitely additive and continuous. That 
is to say, let Q be a set and F an event space of subsets of 0. If Pisa 
mapping from F into [0, 1] satisfying 

(i) P(Q) = 1, P(a) = 0: 

(ii) )if A,B e€ F and ANB=2 then P(AU B) =P(A)+P(B), 

(iii ) if Ay, Ag,...€ F and A; C Aji, for i = 1,2,..., then 


P(A) = lim P(A;), where A = U Aj, 
i- 00 


then P satisfies P(L; Ai) = 2; P(Ai) for all sequences Aj, Ao, ... of disjoint 


events. 


SOLUTION. Firstly, we shall prove that P is finitely additive. Assume 
Aj, Ag € F and Ai Ag = 9, from the hypothesis (ii), we obtain that 


P(A; U Ag) = P(A1) + P(A2), 


and so our statement holds for n = 2. Suppose that the statement holds 
for n = k, that is, 


k k 
P (U a =) P(A), 
i=l i=l 


where Aj, Ao,..., Ay are disjoint events. Ifn =k +1, then 
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(Us) -o(Gsjons) 


























where Aj, Ao,..., Ax, Axyi are disjoint events. Consider the sequence Aj, Ao,... 


of disjoint events. Let By = (ey A;, k © N. The sequence {B,,} of events 
is an increasing monotone sequence, that is By, C Bri, for every k € N. 


We have that 


P (U a = 5 P(Ai) = Spy 


i=1 
where S, is the nth partial sum of the series }°7°, P(A;). The series 
72, P(A;) is a non negative and satisfying 0 < S, = P(Uj_, Ai) < 
P(Q) = 1 so it is a convergence series (Axiom (i)). Using Axiom (iii) 
to obtain that 


°(U-) 


l| 
be 
aw 
ia=u 
g8 
‘Ca 
> 
~~ 


ll 
pay 


| 
i= 
35 
Fg 
, a 
Ce 
> 
Sa 


ll 
an 


= im, Ss" P (Aj) since P is finitely additive 
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PROBLEM (1.11.19). There are n socks in a drawer, three of which are 
red and the rest black. John chooses his socks by selecting two at random 
from the drawer, and puts them on. He is three times more likely to wear 
socks of different colours than to wear matching red socks. Find n. 

For this value of n, what is the probability that John wears matching black 
socks? (Cambridge 2008) 


SOLUTION. The number of ways to choose socks of different colour is 


()("9) =a 


The probability of choosing two socks of different colours is 


3(n—3) 6(n—3) 
(a) ~ n(n—1) 


The probability of choosing two red socks is 





3 6 


(G)C2 n(n — 1)’ 
Since the probability of wearing socks of different colours equals to three 
times the probability of wearing matching red socks, it follows that 





6(n—3) | 6 
n(n—1)  ~ n(n—1)’ 
giving n = 6. For n = 6, the probability of choosing two black socks is 
(2) _ 2:3 





CHAPTER 2 


Discrete random variables 


2.1. Probability mass functions 


EXERCISE (2.8). If X and Y are discrete random variables on the prob- 
ability space (Q,7,P), show that U and V are discrete random variables 
on this space also, where 





U(w) = X(w) + Y(w), V(w) = X(w)Y(w), forw EQ. 


SOLUTION. Because X and Y are discrete random variables, the image 
X(Q) and Y(Q) is a countable subset of R, that is, X(Q) = {x; | i €} and 
¥(9) = {yi [ie N}- 

To show that U is discrete radom variable on (Q., F, P), we need to show 
that: 

(1) the image U(Q) is a countable subset of R, and 

(2) {we0:Uw) =a} eF forz eR. 

Since U(w) = X(w) + Y(w) = UP {ei + Y(Q)} and every set {x; + 
Y(Q)} is a countable set since Y(Q) is a countable set. 

The coutnable union of the countable sets is a countable set, so U(Q) = 
UP {vi + Y (Q) fis a countable set. 

For each x € R, we have 


u(x) = {w € OJU(Q) = x} 
={w Ee Q|X(w)+Y(w) =a} 
{w €O|X(w) =2—-Y(w)} 
LJ Kw €Q\|Xw) =2-t}n {w € Q|YW) = 4]. 
teY (Q) 


EXERCISE (2.9). Show that if F is the power set of 2, then all functions 
which map into a countable subset of R are discrete random variables. 
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SOLUTION. Let X be a function which map 2 into a countable subset 
of R. To show that X is discrete radom variable on (Q,F,P), we need to 
show that: 

(1) the image X(Q) is a countable subset of R, and 

(2) {w EQ: X(w) = 2} € F for ¢ € R.We have {w € 0: X(w) =a} C 
Q and since F is the power set of Q so {w EQ: X(w)=a}e Ff. 

X(Q) is a countable set by hypotheses. 

Thus X is a discrete random variable. 


EXERCISE (2.10). If E is an event of the probability space (Q,F, P) 
show that the indicator function of E, defined to be the function 1g on 2 


given by 
1 ifweb#, 
le= 
0 ifwéeF, 
is a discrete random variable. 


SOLUTION. the image 1p(Q) = {lze(w)|w € |Q} = {0,1} is a countable 
subset of R, and 


lp (2) = {w EN: 1g(w) =2} 
BE ifev=1, 
=< E° ifx=0, 
0 ifaAOandrFl. 


But E ¢ F,E°=O\E € F and § € F so 15'(x) € F for every ER. 
Thus 1g is a discrete random variable. 


EXERCISE (2.11). Let (Q,F,P) be a probability space in which 


0 ='{1;2,3,4, 5,64, PF =4{ 6,424.6}, (13,5 OF. 
and let U, V, W be functions on defined by. 


1 if w is even, 


Rae W(w) =o", 
0 if w is odd, 


U(w) =a, V(w) = 


for w € Q. Determine which of U, V, W are discrete random variables on 
the probability space. 
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SOLUTION. Since U~1(1) = {1} ¢ F, it follows that U is not a disrete 
random variable. 

Note that the image V(Q) = {V(w) : w € OQ} = {0,1} is a countable 
subset of IR, and 


Vo1(2) = {w EN: V(w) = 2} 
{2,4,6} ifa=1, 


ae ed sas) aes ea OP 
a) ifeAOandxzFl. 


But {2,4,6} € F,{1,3,5} € F and @ € F so V-1(z) € F for every r ER. 
Thus V is a discrete random variable. 

Since W~!(1) = {1} ¢ F, it follows that W is not a disrete random 
variable. 


EXERCISE (2.12). For what value of c is the function p, defined by 


He) = {i if k= 1,2)... 


0 otherwise, 
a mass function? 


SOLUTION. The function p is a mass function if and only if 


S/ p(k) = 
k=1 


This is equivalent to 


We have that 
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sO 


= 1 
cye =] 
ee k(k+1) 
1 
= lim (1-2) =] 
noo n 
—> c=1; 


2.2. Examples 


EXERCISE (2.23). If X is a discrete random variable having the Poisson 


distribution with parameter A, show that the probability that X is even is 
—A 
e “cosh A. 


SOLUTION. We have that 


k=0 
Sy ts dees 
2 Bay 
= ope 2k 
k=0 (2k)! 
= e~* cosh X 


EXERCISE (2.24). If X is a discrete random variable having the geo- 
metric distribution with parameter p, show that the probability that X is 
greater than k is (1 —p)*. 
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SOLUTION. We have that 


P(X > K)=1-P(1<X<k) 
= 1 - (pq? + pq’ +--+ + pg 
=1l—-pllt+qt+@t---+4 
1=¢" 


i) 


Re") 








where g = 1—p. 


2.3. Functions of discrete random variables 


EXERCISE (2.26). Let X be a discrete random variable having the Pois- 
son distribution with parameter A, and let Y = |sin(57X yh: Find the mass 
function of Y. 


SOLUTION. Since X has the Poisson distribution with parameter 4, 
then 


Im X = {0,1,2,...}. 


If k = 2m, = 0,1,2,..., then y = |sin $rk| = |sin(mz)| = 0 

If y = 2m+1,m=0,1,2,..., then y = |sin 5rk| = |sin 5(2m + 1)| = 
|sin(Z +mr)| = |(-1)™| = 1. Thus 

mY = {ye Y 14 = |sin 37k] ,k = Ole leseet = {|sin 5kr| , k = 
O21, 2... = 40,1) 
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I 
M 
s 
be 
[© 
= 





= 1—ecoshr 
The mass function of Y is 
e-* cosh A ifx—0 
py) = 1—ecoshr ifx=1 
0 ifeAO0andxFl 


2.4. Expectation 


EXERCISE (2.37). If X has the binomial distribution with parameters 
n and p= 1-—dgq, show that 


























K(X) = np, R(X?) = npq + np’, 
and deduce the variance of X. 


SOLUTION (1). The mean of X is 


























2.4. EXPECTATION 
xXy= So «P(x =z) 
velm X 
= DURP(X =f) 
k=0 
= me kP(X =k) 
k=0 
or ae since K«(%) =n(~ 1) for k > 1 
k=0 
= ike eee 
~ ( : " : 
k=0 
n—-1 | 
= np >> ( , \ig reindex with 7 =k—1 
j=0 \ 4 
= np(p+q)"* 














We have 














Take derivative with respect to p in both sides, we obtain 


2 (7\ 4-1 n—k _ . _ TON fee et. © 
Sok (i) q S_ k(n o(i)e q L2 
k=0 k=0 


since dp = -—1. Thus 
q 
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1 o(n kon-k _ QT Tp m=ke, + 2/7 \ bk nk _ 
ei) TK) ee goer a 


This implies 










































































1 n 1 
-E(X?) — —E(X) + —E(X?) =n 
ed aa ee) 
Dy a, De 5 n 
—— (— + —)E(X*) — —np=n 
P 4 qd 
since (E(X) = np). This implies 
1 2 
Ee) ane 2 
Pq qd 


since p+q= 1. This implies 











R(X?) = npq + np. 





The variance of X 


























var (XxX) = E(X7) _— R(X)? = npq + np? — (np)? = npq 


SOLUTION (2). Consider independence variables 


y= 1 if event A occurs with probabaility P for i =1,2,...,n, 
‘~~ )0. otherwise, 


Thus 
n 
KES Xx, 
i=l 
where 
Therefore 














E(Xj) =O0+g+1-p=p 
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n n 


i=1 i=1 


EXERCISE (2.38). Show that var(aX + 6) = a?var(X) for a,b € R. 


SOLUTION. We have that 


var(aX + 6) =E 








es 














(ax + b) ieee E (aa +b)? 














( 
( 


anxe 


= a? E(X*) + 2abE 
= a E(X?) — 





2abX +b?) — (aE(x) +b)? 
(X) +B? — a? [E(X)]? — 2abE(X) — b? 
a? [E(X)]? 



























































=a? {E(X?) — [E(x)P} 


= a*var(X). 


EXERCISE (2.39). Find 














E(X) and E(X2) when X has the Poisson dis- 














tribution with parameter A, and hence show that the Poisson distribution 
has variance equal to its mean. 


SOLUTION. The mean of X is 
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1 
=e by pa since the term involving k = 0 vanishes 

















co 
k 
=e S- ——__)¥ since the term involving k = 0 vanishes. 


We have that 
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CO 


E(X) =e oa 


k=1 

















=X 


Differentiate this with respect to A to obtain 








= u —A\k . k —r\k-1 
= =" 
aires ares ‘. 


> 

lI 

ua 
ws. 


giving that 




















R(X?) = 1. 








Thus 








tO) = 24 EO) S EM, 




















implying that 





var(X) = E(X?) — [E(X)}? 
=\+N-- 
=. 











2.5. Conditional expectation and the partition theorem 
2.6. Problems 


PROBLEM (2.6.1). If X has the Poisson distribution with parameter 4, 
show that 














B(X(X —1)(X% =2)+ (=k) =O 
for k = 0,1,2,.... 


SOLUTION. The mean of X(X — 1)(X — 2).-(X — k) is 
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co xm 
_ 2A 
r= ae eer 


OO y\i+k+1 








reindex with 7 =m—k—-1 





as required. 


PROBLEM (2.6.2). Each toss of a coin results in heads with probability 
p (> 0). If m(r) is the mean number of tosses up to and including the rth 
head, show that 


m(r) = pl + m(r —1)] + (1 = p)[L + m(r)] 


for r = 1,2,... , with the convention that m(0) = 0. Solve this difference 
equation by the method described in Appendix B. 


SOLUTION. Let H be the event that the first toss gives head. The pair 
H, H° is a partition of the sample space (since exactly one of them must 
occur). Let X be the event that rth head occur . It is easy to see that 














R(X =r|H)=1+m(r-1) for Tr 1s 2 cms 


since if H occurs, then X = r if and only if the first toss is followed by 
exactly r — 1 heads. Similarly, 














R(X =r|H°)=1+m(r) for PH 1 Ds be 
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By the partition theorem, Theorem 2.42, 

















E(X) = E(X | B)P(B) + E(X | BY)P(B*), 
which is to say that 
m(r) = pil + m(r— 1)]) + 1 — p)[L + m(r)] 


for r= 1,2,... , with the convention that m(0) = 0, as required. 
We proceed as follows in order to solve 


























m(r) = pil + m(r— 1)]) + 1 — p)[L + m(r)] 
given that m(0) = 0 and p+q=1. Transfer all the terms to the left-hand 
side 


p[l + m(r — 1)] + (1 — p)[1 + m(r)] — m(r) = 0; 


this is equivalent to 


—pm(r) + pm(r — 1) = —1. (1) 
If the right-hand side of (1) were zero, this would be identical to the homo- 
geneous equation discussed in Example B.7. The new equation is solved in 
two steps. First, deem the right-hand side to be zero and solve as for the 
homogeneous case: 





pm(r) + pm(r — 1) = 0. (2) 


The auxiliary equation of (2) is 


—p+po'=0 
with roots 9 = 1. The general solution to (2) is therefore 
Wt) =o, 
where a is an arbitrary constant. Then, augment this solution by some 


m(r) which has to be given further thought: 


m(r) =a+mi(r). 
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This augmented m(r) has to be such that when substituted into —pm(r) + 
pm(r—1) the result is —1. In this book, it will always be possible to express 
m(r) as the quadratic A+ Br + Cr? with only one of the constants A, B 
and C' non-zero. In the present case try m(r) = Br and therefore require: 


—pB(r) + pB(r — 1) =—1, 


so 
—pBr+pBr—pB=-1. 
Hence 
—pk=B=-1 
So 
1 
B=- 
Pp 
giving 
m(r) =~ 1:0) geal OR ee 
Pp 


is a particular solution to (1). It follows that the general solution of (1) is 


m(r) =a+— fort SL 2s 4 2 


The constant a is found from the boundary conditions to be given by a = 0. 
Therefore the answer is m(r) = r/p. 











PROBLEM (2.6.3). If X is a discrete random variable and E(X2) = 0, 
show that P(X = 0) =1. Deduce that, if var(X) = 0, then P(X = yw) = 1, 
whenever js = E(X) is finite. 





























SOLUTION. Suppsose that X is a discrete random variable and E(X?) = 
0. By (2.28), 


co 
S¢ xfP(X = a) = 0, 
4 =e 
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giving that 


a2P(X = a;) = 0. 
Thus P(X = x;) = 0 for x; 4 0. Hence 





P(X =0) =1-P(X;) 
=1-P( (J {w: X(w) £0}) 


x4, 40,2;E€Im(X) 


=1- So P(X =z) 


x, 40,0;EIm(X) 











as required. 
Let Z = X — E(X). By (2.33), 

















var(X) = E(Z’). 


If var(X) = 0, this is equivalent to E(Z?) = 0, then P(Z = 0) = 1 by the 
argument above. Thus P(X —E(X) = 0) =1. Hence P(X = yu) = 1, where 
p. = E(X) is finite. 



























































PROBLEM (2.6.4). For what value of c and a is the function p, defined 


by 
ck° fork =1,2,..., 
Ho={ 


0 otherwie, 
a mass function? 


SOLUTION. Mass functions sum up to 1, so that 


1= S° p(k) 
k=1 


CO 
=C ) Ke: 
k=1 


giving that a < —1 and c = 1/¢(—a), where ¢(p) = )0,, k ® is the Riemann 
zeta function. 
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PROBLEM (2.6.4). Lack-of-memory property. If X has the geometric 
distribution with parameter p, show that 


P(X >m+n|X >m)=P(X >n) 


for m,n = 0,1,2,.... We say that X has the ‘lack-of-memory property’ 
since, if we are given that X — m > 0, then the distribution of X — m 
is the same as the original distribution of X. Show that the geometric 
distribution is the only distribution concentrated on the positive integers 
with the lack-of-memory property. 


SOLUTION. For the first part, since X has the geometric distribution 
with parameter p (= 1 — q), it follows that 


for k = 1,2,3,.... We have that 





P((X >m+n)N(X >m)) 
P(X >m) 


P(X >m+n 
OS me ie) since m,n =0,1,2,..., som+n2>n 





P(X >m+n|X>m)= 


























= J. reindex with 7 =k-—1andt=i-1 











©9 k 
1l-q : k r 
=— since y — 
q l-r 
1 k=l 





and 
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P(X >n)= S> P(X =i) 


II 
Rs 
= 
i 


=P » ¢ reindex with 7 =i-—1 





Thus 
P(X >m+n|X >m)=P(X > n) 


as required. 

For the last part, let X be the discrete random variable having lack- 
of-memory property and let G(n) be the function on {0,1,2,...} given by 
G(n) = P(X >n). Then 





G(m+n) =P(X >m+n) 
=P(X >m+n|X >m)P(X > m) 
+P(X >m+n|X <m)P(X <m)_ by by the partition theorem, Theorem 1.48 
































=P(X >m+n|X >m)P(X > m) since P(X >m+n |X <m)=0 

= P(X > n)P(X > m) since X has the lack-of-memory property 
= G(n)G(m) 

= G(m)G(n) 


for m,n = 0,1,2,.... Thus 


Px Sk) =O S1))* 
for k = 2,3,.... As a first step towards proving this show that 


P(X > 2) = (P(X > 1))?. 
Define p = P(X = 1) and g=P(X > 1). We now have that 
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P(X >k) =q"*, 


for k = 2,3,.... Since the event {X > k — 1} is the disjoint union of the 
events {X > k} and {X =k}, it follows that 


P(X >k—-1)=P(X >k)+P(X =&), 
which is to say that 


P(X =k) =P(X >k-1)-P(X >k), 


— gh-l_ gk 
=q* (1-4) 
= pg* 
for k = 2,3,.... But for k = 1, 
P(X =1)=p 
=p" *, 


trivially. Hence X takes values in {1,2,3,...} and 


P(X =k) =pq*" for k = 1,2,3,..., 
giving that X has the geometric distribution with parameter p € (0,1). 


PROBLEM (2.6.6). The random variable N takes non-negative integer 
values. Show that 














E(N) = PW > k) 
k=0 


provided that the series on the right-hand side converges. 

A fair die having two faces coloured blue, two red and two green, is thrown 
repeatedly. Find the probability that not all colours occur in the first k 
throws. 

Deduce that, if N is the random variable which takes the value n if all three 
colours occur in the first n throws but only two of the colours in the first 
n — 1 throws, then the expected value of N is 4. (Oxford 1979M) 
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SOLUTION. For the first part, we have that 














SoP(N>k) => SO P(N =1) since P(N >k)= 5° P(N =3) 
k=0 k=071=k+1 i=k+1 
co n—-l 
= os P(N =i) change the order of summation 
i=1 k=0 
=) /nP(N =?) 
i=0 
= E(N), 


as required. 
For the second part, note that the probability that exactly one colour 
occurs in the first k throws is 


Furthermore, the probability that exactly two colours occurs in the first k 
throws is 


To see this, let H be the event that only two colours blue and green occur 
in the first k throws, EF be the event that only two colours green and red 
occur in the first k throws and F be the event that only two colours blue 
and red occur in the first k throws. By the defintion of the probability of 
the union of events (see Exercise 1.20), 


P(HUEUF) =P(A)+P(£)4+P(F) -P(A NE) -P(ANF)-P(ENF)+P(HN ENF) 


i a a ae a 
i a a a 
_ 92k _ (Lye 
=3I(3)" — (3) I}: 


Hence the probability that not all colours occur in the first k throws is 





Ok 
F- Gy =. 


For the last part, we have that 
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E(N) = s P(N > k) 











k=0 
=P(N>0)+P(N>1)+P(N > 2) + P(N > k) 
k=3 


=1+14+1+ 5 P(N >k) 
k=3 


l() -@), 


Hae) a) 




















2\3 1\3 
_ (3) (3) 
=3 ne a 
3 3 
8 9 1 
=3+9 5-57 
8 1 
S34 e.2 
3 6 
_18+16-1 33 11 
a 6 7-6 Be? 


as required. 


PROBLEM 1 (2.6.7). Coupon-collecting problem. There are c different 
types of coupon, and each coupon obtained is equally likely to be any one 
of the c types. Find the probability that the first n coupons which you 
collect do not form a complete set, and deduce an expression for the mean 
number of coupons you will need to collect before you have a complete set. 


SOLUTION. Let A; be the event that we are having all types of coupon 
but cq, 7=1,2,...,c. Then 


PJ A;) = F(A) —SP(AiN Ay) + $5 (ALN Aj) +--+ + (ye) Ai) 
=A i=1 i<j i<j<k 
and 
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for each 7 € {1,2,...,c}. Furthermore, 


P(A;, M Aig M-++ 1 AG,) = cea aa 


for 1<k<ce. Note that 


Ss" P(Aj, N Aig N+ + AG, ) 


i <ig<<ip 


consists of () terms for 1 <k<e. Thus 





a0 A) = @ (c= ae G es VES pana ta Js 


Let N be the random variable which takes the value k if the first k — 1 
coupons do not form a complete set, but the first k coupons are formed a 
complete set. Thus 


SO 
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n=0 
ce-1 oo 

=) °P(N >n)+5 > P(N > n) 
n=0 n=c 














Sere (ee: SE (creo) Vee oe) Ay > \ Se 
ek Oren Sree para (0°) 


1 lfc lfc 2, eal c 
= OF Ged (‘Je z 5(s)e aye sReeenite vA]: 


PROBLEM (*2.6.8). An ambidextrous student has a left and a right 
pocket, each initially containing n humbugs. Each time he feels hungry, 
he puts a hand into one of his pockets and, if it is not empty, he takes a 
humbug from it and eats it. On each occasion, he is equally likely to choose 
either the left or right pocket.When he first puts his hand into an empty 
pocket, the other pocket contains H humbugs. 

Show that if pp, is the probability that H = h, then 


2n—h 1 
n= ( i" ) a= for h =0,1,...,n, 














and find the expected value of H, by considering 


Sis S~ hpn, So (n= h)pn, 
h=0 h=0 


or otherwise. (Oxford 1982M) 


SOLUTION. This is the binomial distribution with parameters 2n — h 
and we may suppose without loss of generality that that the left pocket 
is empty and the right pocket contains H = h. The number of ways to 
take hambugs so that the left pocket is empty is that Gas ", Thus the 
probability pz, is that 
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2n—h 1 
= ( is ) 92n—h for h =0,1,...,n. 
nm 


For the last part, we first prove the formula 


2(n — h) Pp = (2n — h) Phi. 











Indeed, 
2n—h 1 
(2n—h)! 1 
=) h 
in ais — h)! 22n—-h 
_ (Q2n—h)! 1 
~— nl(n — hb)! Q2n-A-1 
(2n —h—1)! 1 
= (2 h 
( n nln = h = 1) PI 
2n—h—-1 1 
= (2n — my( - ) 5anahot 
= (2n— h) Pati 
Thus 
n-1 n-1 
S 5 2(n — h)Ph = S>(2n = h) Pri 
h=0 h=0 
n n n n-1 n-1 
i Pi, — 2npn — 2 a hPr, + 2npy = 2n > Pr, — 2npo — Se +1)Preit So prs 
h=0 h=0 h=0 h=0 h=0 





2n — 2npn — 2E(H) + 2npy = 2n — 2npo — oS hprn +1 — po 
h=0 


(H) 2npo — E(H) +1 — po 
(A) = (2n + 1)po — 1. 


2 

















Gl 





PROBLEM (2.6.9). The probability of obtaining a head when a certain 
coin is tossed is p. The coin is tossed repeatedly until n heads occur in a 
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row. Let X be the total number of tosses required for this to happen. Find 
the expected value of X. 


SOLUTION. Let B;, i = 1,2,...,n, be the event that the first 7 — 1 
tosses give heads and the ith toss gives tails and let B,+41 be the event that 
n tosses give heads. {B,, Bo,...By+1} is a partition of the sample space. 
By the partition theorem, Theorem 2.42, 


























(X) = > PCB) i(X | B;) 


= (1—p)(1+E(X)) + (1— p)p(2+E(X)) +--+ + (1—p)p"™ (n+ E(X)) + p"n 
since E(X | Bj) = E(x) +7 for 7 = 1,2,...,n and E( 
P(B;) = (1—p)p*? for i= 1,2,...n and P(Bn4i1) =p”. Thus 



















































































[L—(1—p)1+p+p*---+p""*)JE(X) = (1—p) [1+ 2p +++) + np" ] + pn. 


Hence 











JE(X) = (1—p)[ptp?+---+p"! +p 











since 1+ p+p?+---+tp™ t= Ee Therefore 


n 









































n pe n . 4 ae 
p"E(X) = (1—p)[p ]+p"n since Sp’ =p 
l-—p = l-p 
[= (n+ 1p" —p)+p-9""" 
=(1 n 
(ip) (ll —p? + p"n 
1— 1)p” n+1 
_1-(n+1)p" + np Lite 
ia 
1-—p’ 
so 
1—p” 
R(X) = ——_—_... 
p'(1—p) 


PROBLEM (2.6.10). A population of N animals has had a certain num- 
ber a of its members captured, marked, and then released. Show that the 
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probability P, that it is necessary to capture n animals in order to obtain 
m which have been marked is 


_afa-i N-a 7 N-1 
EA page=) Net cer n—-1/)’ 
where m <n < N—a+m. Hence, show that 


a f(a-1\(N-a)! XS" — (n-1)\(N— 10)! 
( jaa Ds (n—m)!(N -—a+m-—n)! 


and that the expectation of n is Nii. (Oxford 1972M) 





= 1, 


n=m™M 





SOLUTION. For the first part, let Y;, be the number of animals that it 
is necessary to capture (without re-release) to obtain m, which have been 
marked. For the event {Yn =n} to occur, it is necessary that: 

(i) The nth animal is marked, which can occur in a ways. 

(ii) The preceding n — 1 animals include exactly m — 1 marked and 
nm —m unmarked animals, which may occur in Ce) CS) ways. 

The total number of ways of first selecting a distinct animal to fill the 
nth place, and then choosing n—1 animals to fill the remaining n—1 places 
is N- Ce We Because these are assumed to be equally likely, the required 
probability is 


where m <n < N—a+™m, as required. 
For the second part, we have that 


N-1+m 


> Pn = 1. 
n=m 


Thus 





ay ed (N —a)!(n—1)(N — n)! 
2s 2 ee Se 


Hence 
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a (a—-1\(N—-a)! XS” )— (n-1)(N— nn)! 
=(4 1) = (n—m)'(N—a—n+m)! 


n=mM 


as required. 
For the last part, we see from (1) that 


N-a+m 


’ 














(n-I(N-n)!  _ N(m-D(a-m!(W- 
= (n-m)\(N-a-n+m)! a (a—1)! 
7 N\(m — 1)!(a — m)! 
~  al(N —a)! 
= (") (m—1)'(a—m)! 
Thus 
N-a+m ae LN — n)! : 
= (n—m)'(N-a—n+m)!(m—1)i(a—m)! 


ae) 


n=m™M 


Applying (2) for N + 1, a+1 and m +1 gives that 


pps (7) (as) = Ce ? 


The mean of 7 is 
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N-a+m 
(n)= SS) np 
-£ (07! Giger eee nl(N —n)! 
ON ni) Wo = (n-m)\(N-a-—n+m)! 
_ a Nag ni(N —n)! 
=m(*) NI Dy (n—m)\(N —a—n+m)! 








= (6) ee () Tn) 











CHAPTER 3 


Multivariate discrete distributions and 
independence 


3.1. Bivariate discrete distributions 


EXERCISE (3.8). Two cards are drawn at random from a deck of 52 
cards. If X denotes the number of aces drawn and Y denotes the number 
of kings, display the joint mass function of X and Y in the tabular form of 
Table 3.1. 


SOLUTION. We have that 











(1G) _ 44! 4! also! 88 
=) ~ 1143! 1138! «52! 13-51’ 
(S) 44! 2150! 11-43 

pxy {a = 0,y = 0) (2) ~ 21421 52) 13.51” 
_@-G@)_ 4) 44! 2tso! 88 
= ~ 113! 11438! 52! 13-51’ 
GG _ 4! 4! 20! 8 

113! 113! 52! 13-51’ 























() 4! abo! 1 
Ot Got = 1a ele 











Thus 
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1-43 BB T 
y=0] 7 gl isl 13.17 
yal 1351 13:51 
ES 13-17 0 0 


TABLE 1. The joint mass function of the pair X,Y. 


EXERCISE (3.9). The pair of discrete random variables (X, Y) has joint 
mass function 
Bee St ag S02 
PX aay =4)= 1 tJ rene sere) 
0 otherwise, 
for some value of @. Show that 6 satisfies the equation 
6 + 267 + 36° + 2604 + 6 =1, 
and find the marginal mass function of X in terms of 6. 


SOLUTION. We have that 


SOP(X =1,¥ =f) =04+0+0474+0R +0404 0+0° 
J 





=6 +267 + 30° + 2074+ 6°. 
But 


pxy(x,y) =1, 


x€Im X yeImy 
SO 


6+ 207 + 30° +264+6=1 


The marginal mass function of X is 
px(«) = >> pxy(2,y), 
y 


giving that 


px (0) =pxy(«# =0,y = 0) + pxy(e =0,y = 1) +pxy(e4 =0,y = 2) 
=6+697+6°. 


Thus 


for 7 = 0,1, 2. 


3.2. EXPECTATION IN THE MULTIVARIATE CASE 


px (i) = girl oh pit? A gts 
= 0(0+ 6? + 6°) 


3.2. Expectation in the multivariate case 


EXERCISE (3.12). Suppose that (X,Y) has joint mass function 


Show that 


and 


PXS=ZY =7)=e" 


























SOLUTION. We have that 














«=0 y=0 


E(XY) = 6° + 464 + 40° 


E(X) = 6? + 36° + 36* + 20°. 


2 2 
(XY) =) > (ey) P= 2, Y Sy) 


for 7,7 =0,1,2. 


= (1-1)P(X =1,Y =1)+(1-2)P(X =1,Y =2)) 
£2 DP eS oY 1) + 22x =o 7 =2 
= 19° + 264 + 264 + 46° 


= 9° + 494 + 46° 


Note that (47)P(X =1,Y = 7) =0 for i =0 or j = 0 and 














z=0 y=0 


=] 


P(X =1,Y =0)+ 1 





2 Z 
R(X) = S00 aP(X =2,Y =y) 








= 1967 + 16° + 164 + 26° + 204 + 20° 
= 9 + 36° + 364 + 26° 
Note that P(X =i, Y =j) =0 fori =0. 


MAS Y= 1) IPC yY 
+ 2P(X = 2,Y =0)+2P(X =2,Y =1)+2P 


X=2,Y =2 
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3.3. Independence of discrete random variables 


EXERCISE (3.21). Let X and Y be independent discrete random vari- 
ables. Prove that 


P(X >axand Y >y)=P(X > 2)P(Y > y) 
for allz,y ER. 
SOLUTION. Let {Im X > x} = {21,22,...} and {ImY > y} = {y1, yo,...} 


P(X >aandY>y)= S> P(X = x; and Y = y;) 
ij=l 














(oe) co 
= P(X S4;) Ss" P(Y = y;) by independence 
ijl ij=l 


=P(X >2)P(Y >y) 


EXERCISE (3.24). The indicator function of an event A is the function 


14 defined by 
1 ifweA 
1 a ‘ 
A) ‘s ifw Z A. 


Show that two events A and B are independent if and only if their indicator 
functions are independent random variables. 


SOLUTION. Suppose that X and Y are independent, then P(AN B) = 
P(A)P(B). 





Pia,1p (0,0) = P(A BY) 
= P(A‘)P(B°) (since A and B are independent so are A and B°) 


= pi, (0)p1,, (0) 














Piatp(1,0) = P(AN B*) 
= P(A)P(B°) (since A and B are independent so are A and B°) 
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Piajtp (0,1) = P(ATN B) 
= P(A‘)P(B) (since A and B are independent so are A° and B) 
=Pi, (0)p1, (1) 














Piaip(i,1) = P(AN B) 
= P(A)P(B) 
Pia (1)pi, (1). 











Thus 
Pitaig) = Pi, (t)Pip (7) 
for 7,7 = 0,1. Hence 1,4 and 1g are independent random variables. 


Conversely, suppose that 1,4 and 1g are independent random variables. 


We have 








Pigip(1,1) = Pas = 1,13 = 1) 
=P(ANB). 








But 











Piatp(ls 1) = pr,()pig(1) = Pa = 1)P(1e = 1) = P(A)P(B). 
Thus 
P(AN B) = P(A)P(B). 
Therefore A and B are independent. 


EXERCISE (3.25). If X and Y are independent discrete random vari- 
ables, show that the two random variables g(X) and h(Y) are independent 
also, for any functions g and h which map R into R. 


SOLUTION. For a,b € R, we have 














P(g(X) =a, h(Y) = 6) = P(X =g"'(a),Y =h*(b)) 
= P(X =g"‘(a))P(Y =h71(b)) since X and Y are independent 
= P(g(X) = a)P(A(Y) = 8) 


Thus g(X) and A(Y) are independent. 


84 3. MULTIVARIATE DISCRETE DISTRIBUTIONS AND INDEPENDENCE 


3.4. Sums of random variables 


EXERCISE (3.29). If X and Y are independent discrete random vari- 
ables, X having the Poisson distribution with parameter A and Y having 
the Poission distribution with parameter 4, show that X + Y has the Pois- 
son distribution with parameter \ + yu. Give an example to show that the 
conclusion is not generally true if X and Y are dependent. 


SOLUTION. By Theorem 3.27, 


= yy P(X =2)P(Y =z-2) 


x€Im X 


= $7 P(X = 2)P(Y = 2-2) 
xz=0 





= P(X =0)P(Y = z)+ P(X =1)P(Y = 2-1) 
+ +P(Z = z)P(Y =0) 











1< zl 
et > AT 2 eT OATH) 
2G al(z — 2)! 


=i z (o) ate) 
z=0 


1 
= Kiara! 
yas 


since z— xz > 0, 


sox<zandzx>0 


multiplying and dividing by z! 


since (A+ y)* = SS () NE Ee 


x=0 


Therefore X + Y has the Poisson distribution with parameter \+ yu. Take 
Y = X, so that X + Y takes even values only—it cannot then have the 


Poisson distribution. 


EXERCISE (3.30). If X has the binomial distribution with parameters 
m and p, Y has the binomial distribution with parameters n and p, and 
X and Y are independent, show that X + Y has the binomial distribution 


with parameters m+n and p. 
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SOLUTION. By Theorem 3.27, 


--+P(Z = z)P(Y =0) 

(C)ha-nm (Zea mts (T)eta-prr(,")emta as 
+ (Tera —n°(o) haa 

(“ora 2 

ie cc 


We shall prove that )>7_) (")(,”,) = (a). Indeed, we have a group 
of people of m men and n women, the number way to choose z people out of 
group is that )>*_ ("")(,",,) but this also equal to ae This complete 


z 
tproof of the identity. Thus 


pr(1 
n 
= ym z-k (4 n—(z—-2) 
P) ( "ye (1 —p) 
m+n—(z—2) 


p (1—p) 


x 
x 


m+n z Ti-rtt— (2s 
PZ =2)=( . )r*0-p) tee), 


hence Z = X + Y has the binomial distribution with parameters m+n and 
p. 
For n = 2, by Theorem 3.27, we have 








P(X + Xa=0) = PUG = O(a =0) = a= = (F)e- v= 1-9) 


P(X, + Xq = 1) = P(X = 0)P(X2 = 1) + P(X = 1)P(X2 = 0) = 2pg = ( 
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2 


Thus X; + X29 has the binomial distribution with parameters 2 and p. 
Suppose S, = X; + X9+---+ Xy has the binomial distribution with 
parameters n and p. That is, 


P(X; + Xp =2) = P(X 1)P(X=1) =? = (39204 


P(Sn =) = (7')p*0 pyr 


where k = 0,1,...,n. Consider S,41 = X, + Xo+---+Xn4+Xn41 = 
Sn tb Angi 


P Saaz) = > P(Sn = 2) P nar 2 2): 
x=0 


We have z-—x=Oorz—x=1,sor=zorr=z-—1,s0 


P(Spu1 = 2) =P(Sn = 2)P(Xai = 0) +P(S, = 2— UP(Xpa1 = 1) 
yil~p)*(l=2) + (. " ea — py Vp 


Z(1 _ m\n—(z—-1) Z(q _ n\n—(z—-1) 
p(1—p) +("\)o (l—p) 


z 
n n Z(1 _ m\n—(z—-1) 
(" +(",)]ea P) 
_ @ + pF (1—p)""@-) (by Pascal’s Identity) 
z 


Thus S,,4; has the binomial distribution with parameters n+ 1 and p. 
By induction, S;, has the binomial distribution with parameters n and 
p for alln EN. 


3.5. Indicator functions 


EXERCISE (3.42). Let N be the number of the events Aj, A2,...,An 
which occur. Show that] 


1A similar fact is valid for an infinite sequence Aj, Ao,..., namely that the mean 
number of events that occur is }*7°, P(Ai). This is, however, harder to prove. See the 
footnote on p. 40. 
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SOLUTION. Since 


1 if A; occurs 
la, (w) = 


0 otherwise 
it follows that 


so that 


3 
3 


























E(N) = ))E(1a,) = > P(A) 


i=1 i=1 
3.6. Problems 


PROBLEM (3.6.1). Let X and Y be independent discrete random vari- 
ables, each having mass function given by 


P(X =k) =P(Y =k) = pq fork =.0, 152. ge 
where 0 < p=1-—q <1. Show that 








1 


SOLUTION. We have that 


P(X =k,X+Y =n) 





P(X =k| XY =n) = 





P(X +Y =n) 
_ P(X =kh)PY =n —k) 
7 P(X +Y =n) 


PIX=jh)PY H=n—k) = pot pg” "=p" 


P(X +Y =n) = Wop P(X =h)P(Y = n—k) = Vo apg * = 
ae oe = (n 4 1)p2q” 
Thus 
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P(X =k| X+Y¥ =n) = (BGs = gh fork =0,1,2...,n. 


PROBLEM (3.6.2). Independent random variables U and V each take 
the values —1 or 1 only, and 


PUL =) =a, PV =1)=8; 
where 0 < a,b < 1. A third random variable W is defined by W = UV. 
Show that there are unique values of a and 6 such that U, V, and W 


are pairwise independent. For these values of a and b, are U, V, and W 
independent? Justify your answer. (Oxford 1971F) 


SOLUTION. Since independent random variables U and V each take the 
values —1 or 1 only, so the third random variable W = UV also takes the 
values —1 or 1 only. 

P(W = -1) = PU = -1)P(V = 1)+ P(UU = 1)P(V = -1) = 
(1—a)b+a(1— 6b) =a+b-— 2ab 

P(W =1)=P(U =1)P(V =1)+ PU =-1)P(V = -1) =ab+(1- 
a)(1—b) = —a—6+2ab+41 





P(U =—1,W =-1) P(U = -1,UV =-1) 

= PU=-1,V=1) 
P(U = —1)P(V = 1) (since U and V are independent) 
(1—a)b 




















and 


P(UU =-1)P(W=-1) = (1—a)(a+b6-—2ab) = 207b- a? —3ab+a+b 
by Definititon 3.13 


(l—a)b = (1-—a)(a+b- 2ab) 
b = a+b-—2ab (since 1— a £0) 
This implies 6 = 5 





P(V=-1,W=-1) = P( 














= —-1)P(U =1) (since U and V are independent) 
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and 


P(V =-1)P(W =-1) = (1-—5})(a+b- 2ab) 
by Definition 3.13 


(l—bja = (1—6)(a+b-— 2ab) 
a = a+b-—2ab (since 1 — b £0) 


This implies a = 5: Plug a=b= 5) we see that U, V, and W are pairwise 
independent. We have 
































1 
P(U = 1)P(V = 1)P(W = 1) = ab(—a — 6+ 2ab4- 1) = 3 
and 
PHY Siw ST) MU =1V =1,0V = 1) 
PU Sy 1) 
=P( =1)P(V=1) 
1 
=ab= 4 
Thus 








P(U =1)P(V = 1)P(W =1) 4P(U =1,V =1,W =1) 


fora=b= 5: Therefore for these values of a and b, U, V, and W are not 
independent. 


PROBLEM (3.6.3). If X and Y are discrete random variables, each tak- 
ing only two distinct values, prove that X and Y are independent if and 


only if EXXY) =E(X)E(Y). 
































SOLUTION. The necessity of the condition is immediate from Theorem 
3.19. To prove sufficiency, suppose X are discrete random variables taking 
only two distinct values x; and x2, and Y be discrete random variables 
taking only two distinct values y; and y2 ,and let E(XY) = E(X)E(Y). 
Then 
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E(XY) = ayyiP(X = 21,Y =y1) + ti1y2P(X = 21,Y = yo) 
+ xoyiP(X = 22, Y = y1) + Loy2P(X = 22, Y = yo) 























E(X)E(Y) = [xi P(X = 21) + 2oP(X = 22)| [mP(Y = y1) + wP(Y = y2)] 
=497 (PH ay Hn) FP Sai = ya) 

+ x [P(X = 22, Y =y1) + P(X = 22,Y = yo)|} 

{yi [P(X =21,¥Y =y1)+ P(X =22,Y =y)| 

+ y2 [P(X = 21, Y = yo) + P(X = 22,Y = y2)|} 

= {eX Sr Ya) eee aaa Y Sap) 

+ xoP(X = 22, Y = y1) + roP(X = 12, Y = yo)} 

{yi P(X =271,Y =y1) + yiP(X = 2%,Y = y1) 

+ yoP(X = 21, Y = yo) + yoP(X = 22,Y = y2)} 

giving by E(XY) = E(X)E(Y) that 







































































L1YiPx y(L1, y1) + Tiyopxy (@1, y2) + cayipx,y (2, y1) + coyopx y (La, y2) 
= [v1px (1) + £epx (x2)] [yipy (y1) + yepy (y2)] 
The joint mass function of the pair X, Y. 











x1 | x2 
yr | al b 
y2| c | d 

















wherea+b+ct+d=1 


Lyyia + Lyy2e + Loy1b + Loyod 
=r1y1(a+ c)(a +b) + 21y2(a + c)(c+ d) 
+ xoyi(b + a)(a + b) + roy2(b + d)(c + d) (*) 


We shall prove that if E(XY) = E(X)E(Y), then E ((aX + B)(yY + 6)) = 
E(aX + B)E ((vyY +6)) where a, 6,7,6 € R. Indeed, 
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E((aX + B)(vyY + 6)) = ayE(X,Y) + adE(X) + ByE(Y) + By 
= ayE(X)E(Y) + ad E(X) + BY(Y) + Bd 
= E(aX + B)E(yY +0) («x) 


Define X; and Y, by 
xX — Y — 
oe v2 ae Yo 
v1 — XQ Yi — Y2 
Then the joint mass function of the pair X,, Y1 is 











Apply (*) and (**), we have 


a=(a+c)(a+b) 
Thus P(X = 21,Y =y1) = P(X =21)P(Y = yo). Then using 














TQ — X41 : yo— Yi 
we have P(X = 2%2,Y = yo) = P(X =22)P(Y = ye). Using 














X — £2 Yow 
X3=——— 3=9Y=——, 
X1 — XQ y2— V1 
and 
X—2\ Y-y, 
X4= Y4 = a 
LQ — XL] y2— V1 


we have the remain formulas. So X and Y are independent random vari- 
ables. 


PROBLEM (3.6.4). Let X1, X2,..., Xn be independent discrete random 
variables, each having mass function 


1 
P(Xi =k) = for k=, 2pcy Ve 


Find the mass functions of U, and Vy, given by 


92 3. MULTIVARIATE DISCRETE DISTRIBUTIONS AND INDEPENDENCE 


Up = min Xp Mas 4 Nak Va ae Moe, Ay b 
SOLUTION. For k = 0,1,2,...,N, we have 
n 
= [[®@: > k) by Exercise 3.23 


k-1 
k-- 
ee i ince P(X; =S° P(X; =j) = —— 
since P(X; <k) xP j= 








2 Raa, (KE) = Ty 
a — yr) - 
Se ktay 

= (l= y-1- 


ot 


for k = 0,1,2,...,N. It is clear that if k > N +1, then P(U;, = k) = 0. 
For k = 0,1,2,...,N, we have 


P(Vn Sk) = P(X1 Sk, Xo Shy...) Xa Sh) 

= [[®: <k) (by Exercise 3.23) 
k-1 
= eae (since P(X; < k) = So P(X: = j) = —) 


jel 


Moreover, 


3.6. PROBLEMS 93 


P(Va=k) = P(Va<k)—-P(Vn>k-D 


= Ier-fer 
k” —(k—1)" 
Bg aie) 


for k = 0,1,2,...,N. It is clear that if k > N +1, then P(V;, =k) = 0. 


PROBLEM (3.6.5). Let X and Y be independent discrete random vari- 
ables, X having the geometric distribution with parameter p and Y having 
the geometric distribution with parameter r. Show that U = min{X,Y} 
has the geometric distribution with parameter p+ r-— pr. 


SOLUTION. Let {U =k} ={X =k, Y >k}U{Y =k, X >k}. Then 


P(U =k) =P(X =k, Y >k) +P(Y =k, X >k) —PU{X =k, Y > kh} N{Y =k, X > k}), 





giving 
do P(X = k)Y = 5) 
j=k 
+S> PY =k,X =j)—-P(X =k,Y =k) =pq" rs! thors ce a a — pq Ts 
j=k j=k j=k 





— pr[(l—r)(1 — p)| 
=pl(l—p)—r)P" +r[(1-p)a—r)P 
~ pr[(l—p)(1— rh" 
=(p+r—pr)[(l—p)—r)P". 
Let s = p+r-—pr, thn 1—s =1—p—r+pr=(1-—p)(1-,7r). So 
P(U =k) = s(1—s)*~!, therefore U has the geometric distribution with 
parameter s =p+r-—pr. 








PROBLEM (3.6.7). Let X1, X9,...be discrete random variables, each hav- 
ing mean p, and let N be a random variable which takes values in the 
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non-negative integers and which is independent of the X;. By conditioning 
on the value of N, show that 


























H(Xy + Xo+---+Xy) = wE(N). 


SOLUTION. Let B; be the event that B; = {w : N(w) = i— 1} for 
i1=1,2,.... Then B,’s form a partition of the sample space for 7 = 1,2,.... 
Define Sy = X, + Xo+---+ Xy. Then 


(Sw) = S- E (Sv | Bi)P(Bi) 


























We have that 


( 
= E(X, + X2+---+Xi-1) 
= E(X1) + E(X2) +--+ + E(X-1) 
= (i - 1) E(X1) 
=(i-1)p. 


Thus 











E(Sw) = S_(- 1)uP(N =i-1) 








=n S7(i- P(N =i-1) 


= pE(N). 

PROBLEM (3.6.8). Let X1, X2,... be independent, identically distributed 
random variables, and S, = X; + X2+---+Xy. Show that E(S,,/S,) = 
m/n if m <n, and E(S,/S,) = 1+ (m—n)pE(1/S,) if m > n, where 
p = E(X1). You may assume that all the expectations are finite. 


















































SOLUTION. For m <n, then 





Sm Xi +Xot---+Xm 
E =E 
(F) =F + ) 

A n & 

=) E(=) 
i=l Sn 

X1 ; Xi : : ae ; 
= mE(—) since — are identically distributed random variables. 


Sn Sn 
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= 
“——~ 
at 
we 
7 
I 
Sle 


In particular, m = n, then E(1) = 1 = nE( +). This implies E(=+ 
Thus 

























































































(2m) = ™ 
n n 
For m > n, 
Sm Sy “1 ue Xj 
Ca (s+ u( + 3) 
i=n+1 
= E(1) + LS (X;) p) since X; for i >n-+ 1 is independent with 1/5, 
i=nt1 o 
= 1+ (m= n)uB(>) 
= mn) ae): 


PROBLEM (3.6.9). The random variables U and V each take the values 
+1. Their joint distribution is given by 











P(U =+1)=P(UU =-1)=., 








1 
PV =+1|U=+1)=3=P(V=-1|U=—1), 


2 
P(V =-1|U=1)=5=P(V=41/U=-1). 








(a) Find the probability that x? + Ux +V = 0 has at least one real 
root. 

(b) Find the expected value of the larger root, given that there is at 
least one real root. 

(c) Find the probability that 2? + (U+V)a+U+V =0 has at least 
one real root. 


(Oxford 1980M) 


SOLUTION. (a) 2? +Ux+V =0 has at least one real root if and only 
if U2 —4V > 0. Since V = +1, it follows that U? — 4V > 0 if and only if 
V=-1. 





i ee 
32°32" “2 
Thus the probability that 2? +Uaz-+V =0 has at least one real root is 

2 








P(V =-1) =P(V =-1|U =1)P(U = 1)4+P(V = -1| U = -1)P(U = -1) 
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(b) Let G be the random variable that taking the value is the larger 
root. 


If U =-1 and V =-—1, then G = 148 








14+ V5 
P(G= ws) PH tat 
= P(V=-1|U=-1)P(U =-1) 
2) 4h a 
— 3 2 6 
If U =1 and V = -1, then @ = =14¥5 
=1'.4/5 
pg= 2b) =, PU avo 
= P(V=-1|U=1)P(U =1) 
ee ae ee 
= a. Or 3 








E(G) = Live P(G = V5) V8 P(g = aev5) = i+v8.1 as, 
1_ V5 _ 1 _ 3V5-1 
2 6 . 


3 6 

(c) Let W =U+V. Then 2? + (U+V)r2+U4+V=0 => 2? 4+ 
Wa+W =O, and hence 2? +(U+V)x+U+V =0 has at least one real 
root if and only if 2? -+Wa+W =0 has at least one real root. This means 
W?—4W > 0. Since W = U+V, then W takes values 2, 0,—2. Thus 
W?-4W >0 — W=0orW =-2. 

P(W =0) = PU =-1,V =1)+ PU =1,V=-1)=P(V=1| 
U =-1)P(U = —-1)4+ PV =-1|U =1)P(U =1) 

ae ee wie ee eee 

3.2 3. 2 3 
PW] 22) = POS Sg Sel) = PV SH | eS Pe 


















































P(W =0or W =-2) = P(W =0)+ P(W = 2)=24+},=2. 


PROBLEM (3.6.10). A number N of balls are thrown at random into 
boxes, with multiple occupancy permitted. Show that the expected number 
of empty boxes is (M — 1) /MN7-1. 





SOLUTION. Let Z; be the indicator function that the ith box is empty. 
The total number of empty boxes is 
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S=4+Zo+-:-+Zy. 
Also, 














Since the probability that one particular ball does not land in the ith box 
is (IM — 1)/M, so the probability that all N balls do not fall in ith box is 
this 


so that 


























by symmetry. 


PROBLEM (3.6.11). We are provided with a coin which comes up heads 
with probability p at each toss. Let v1, v2,...,Un be n distinct points on 
a unit circle. We examine each unordered pair v; , vj in turn and toss the 
coin; if it comes up heads, we join vj; and v; by a straight line segment 
(called an edge), otherwise we do nothing. The resulting network is called 
a random graph. 

(a) the expected number of edges in the random graph is $n(n — 1)p, 

(b) the expected number of triangles (triples of points each pair of which 
is joined by an edge) is ¢n(n — 1)(n — 2)p®. 


SOLUTION. (a) Let N be the number of edges in the random graph, 
and A;; be the event that there is an edge between v; and v;. Then 


N= Ss" ee 


1s<ifjen 


so that 
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E(N) = Ss" E(14,;) 
= Pg) 











there are (3) terms in the sum, 


P(Aij) =p 


II 
“~~ 
Nw 3 
NS 

3 


1 
<n(n—1)p. 
5 iu(n )p 


(b) Let M be the number of triangles in the random graph, and Bj;z 
be the event that there is a triangle v;v;v, . Then 


Mee Se «dares 

















1<iAjAk<n 
so that 
E(M) = E(12;5,) 
1<idj¢k<n 
= (Bizz) 
1<ifAjAk<n 
7 (") 3 there are (3) terms in the sum, 
Bue P( Bie) = P(A N Aik O Ajax) = P(Aiz)P(Aiz)P(Ajx) = 
1 


= grin —1)(n—- 2)p°. 


PROBLEM (3.6.12). Coupon-collecting problem. There are c different 
types of coupon, and each coupon obtained is equally likely to be any one 
of the c types. Let Y; be the additional number of coupons collected, after 
obtaining 7 distinct types, before a new type is collected. Show that Y; has 
the geometric distribution with parameter (c — 7)/c, and deduce the mean 
number of coupons you will need to collect before you have a complete set. 


SOLUTION. Let N be the number of coupons you need to collect before 
you have a complete set. then 


W2TEY VME 
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The probability of choosing a new coupon other than 7 obtained distinct 
types is as The probability of getting (k — 1) consecutive coupons be- 
longing to 7 obtained distinct types is ()R1. The probability of getting k 
additional coupons collected after obtaining 7 distinct types, before a new 
type is collected 





C4 4 
P(¥j=k) = -)e4 
=k) = 4) 
for k = 1,2,3,.... Thus Y; has the geometric distribution with parameter 


p= (c—1)/c and i/e =1—(c—1)/c=1-—p. We have 














E(Y;) = kP(Y; =k) 
kel Y; 
=) kP(Y) =k) = So kP(Y; =F) 
k=1 k=1 





I 
me 
QO 
io) 
wS 
“—— 
als. 
~" 
- 
a 
| 
QO 
io) 

BS 
——~ 
als 
~" 
7 
e 











k=1 k=1 

_ cad adh ba 

ala mul) where u = j/c 

= ee ed 1 629 1 2-8 
~~ ¢ dul-uw ec (1-4)? c (1—4)2 e-j 


Thus 
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E(N) =E(14+%+¥o+---+¥e-1) 
= E(1) + E(Y1) + E(Y2) ab vee Et Y. 1) 




































































I 

KH 

+ 
ae 
bg 





I 
+ 
ie) 

“——~ 
~~ 





PROBLEM (3.6.13). In Problem 3.6.12 above, find the expected number 
of different types of coupon in the first n coupons received. 


SOLUTION. Let N be the number of different types of coupon in the 
first n coupons received. Let A; be the event that at least one coupon of 
type ith is received, and let A be the event that there is no coupon of type 
ith in the first n coupons received. Then 


N= 5 kas 
i=1 


so that 


























E(N) =) E(14,) = 5) P(As) = P(Ai) = el — P(AS)], 
i=1 


by symmetry. It is easily seen that P(A{) = (1 —1/c)”, and hence E(N) = 
c{1 — (1 — 1/c)"] regardless of the value of n. 














PROBLEM (3.6.14). Each time you flip a certain coin, heads appears 
with probability p. Suppose that you flip the coin a random number N 
of times, where N has the Poisson distribution with parameter A and is 
independent of the outcomes of the flips. Find the distributions of the 
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numbers X and Y of resulting heads and tails, respectively, and show that 
X and Y are independent. 


SOLUTION. Let B; be the event that {N = 7} fori =0,1,2,.... Then 
B;’s form a partition of the sample space. Let X be the numbers of of 
resulting heads. It is easy to see that for m = 0,1,2,..., 


n 
k; 








P(X =k| B;)=P(X =k| N =i) (j)a-2 for k= 1,2)... 


since N is independent of the outcomes of the flips. Moreover, since N has 
the Poisson distribution with parameter 4, 


1. 
P(N =i)==ANe*  fori=0,1,2,.... 


By the partition theorem, Theorem 1.48, 
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= a i- tor 
=> (;,)oKa pve 
i=k 
2! = a! k eee 1—AX 
= aga? a eae 
= se l pr(l py Ne 
2+ kG — by)! 
_ Pp —X = 1 Lyi 
“Hog ecg 
Pp —r = 1 j+k y\j+k 
= —(1 — p)itk3 
kL = pyr dX i ) 
k °° j 
P —r Cray 
=e oe 7 
k\l—p)® se 
= pe —d ,(1—p)d 
K(— pk * 
= Di ep 
kl(1 — p)* 


We have 
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ce 


Ne AP yi Ke A(1—p) 


co /. pik 
e >? (pr) Fe *0-P) S- (;) (1 ») 














i=k 
o0 : i-k 
— pr k .—A(1—p) a (1 =p) 
Sean) 2 kG—k! al 
i=k 
—»p Op)* -aa—p) > (1 - p)* 
_ Ap \AP (1—p) P 
mie k! le d (i—k)! 
a ae 
=e? wv) AP) DS u “H reindex with 7 =m—k 
j=0 
k O91 
= 7? AP)" .-A1-p) -A1-P) use Maclaurin series > eae = e* 
k! = | 
= en? p)* 
k! 


Hence, X has the Poisson distribution with parameter Ap. 
Let Y be the numbers of of resulting tails. It is easy to see that 


since tails appears with probability 1 — p and the roles of the appreance of 
a head and a tail are similar. Hence, Y has the Poisson distribution with 
parameter A(1 — p). 

We have 
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RX ERY js a PX any js NSP) since N=X+Y 
P(X =r|N=r+s)P(Y =s|N=r+s) 
r+s x s—AXA are 
= 1 
Oe ea, 
_ (r ar s)! r roa are 
ris! eye (r+)! 
= (0p) 1G = 9 
r! s! 


Hence, X and Y and Y are independent random variables. 


PROBLEM (3.6.15). Let (Z, : 1 <n < co) bea sequence of independent, 
identically distributed random variables with 


PZ, = 0) =; P(Zn = 1) =p, 


where p+q=1. Let A; be the event that Z; = 0 and Z;_; = 1. If Up, is 
the number of times A; occurs for 2 < i <n, prove that E(U,,) = (n—1)pq, 
and find the variance of U;,. (Oxford 1977F) 














SOLUTION. For 2<i< 7 <n, we have 


P(A;) = P(Z; =0,Zj1 = 1) 
( 


= Pi ZO) Pee = Jt) by independence 
= dp 

= P(Z; =0)P(Zj-1 =1) 

= P(4;=0,23-4 = 1) by independence 
= P(A5) 


Hence P(A;) = P(A;) = pq, for 2<i<j <n. Thus 
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and the variance of U,, is 








vat(U,) = E(U?) — E(U,,)” 















































In 
= S42 DS dada 
2NI<j 
-Yols +2 S lain; since l4,la, = Lajna, 

2 

Therefore, 
—3)(n—-2 
(U2) = (ta +2 $0 Laina;) = (n—-1)p4 pol” as )P(AWNAy) 
2I<j 


for j >i+2. Since A; 7 Aj41 = @ and there are (n — 3)(n — 2)/2 terms 
E(14,na;) for i < j+2 in the second sum and P(A;,1.Aj,) = P(Ai,A;,) = 
2¢@? for j, > i, +2, jo > ig +2. Thus 


























E(U2) = (n — 1)pq + (n— 3)(n — 2)P(Z; = 0, Zi_-1 = 1, Z; = 0, Z}-1 = 1) 
= (n — 1)pq + (n— 1)(n — 2) P(Z; = 0) P(Z; = 0)P(Zi_1 = 0)P(Zj;-1 =0) for j>i+1 








Therefore 
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var(U,) = E(U2) — E(U;,)" 

pq + (n — 3)(n — 2)p*q? — [(n — 1) pq)? 

pq + (n — 3)(n — 2)p*q? — (n — 1)*p?q? 

pq + [n? — 5n + 6 — n? + 2n — 1]p?q? 
= (n — 1)pq — (3n — 5)(pq)” 

PROBLEM (3.6.16). I throw two dice and record the scores $j and Sp. 
Let X be the sum $; + S2 and Y the difference S; — So. 

(a) Suppose the dice are fair, so that the values 1,2,...,6 are equally 
likely. Calculate the mean and variance of both X and Y . Find all the 
values of x and y at which the probabilities P(X = x), P(Y = y) are each 
either greatest or least. Determine whether the random variables X and Y 
are independent. 

(b) Now suppose the dice give the values 1,2,...,6 with probabili- 
ties p1,p2,---,pe and qi, q2,---,96, respectively. Write down the values 
of P(X = 2), P(X = 7), and P(X = 12). By comparing P(X = 7) 
with \/P(X = 2)P(X = 12) and applying the arithmetic/geometric mean 
inequality/| or otherwise, show that X cannot be uniformly distributed on 
the set {2,3,...,12}. 

(Cambridge 2009) 


SOLUTION. We have that 




















WN SS 


( 
=(n- 
( 


n—- 





























1)/2/3) 4/5 )]6 
1] 2/3/4/ 5] 6] 7 
2).3)/4/5])] 6] 7] 8 
3]/4)/5]/6] 7} 8 | 9 
4]75/6)7) 8 | 9 | 10 
5|}6)7])8) 9 | 10) 11 
6 |) 7) 8]9 ] 10) 11) 12 
































10. 1 
6 6 36 


2See the forthcoming Example 7.70 also. 
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BX =i) = S$) PS = pP(S=3) 


i+j=10 


36 






































itj=12 
= 3 
36 
Thus 
12 
R(X) = 5” aP( 
r=2 
3 5 6 
= 2 2 + 4 Fee NG 7 
36° 36 36, 36 36. | / 36 
18%. OS thie 4 10. 2 £44 e es Be, z 
36 36 36 36 - 36 
= 7 
variance 
12 
E(X*) = Soa? P(X =2) 
x=2 
1 2 3 4 5 6 
as es a ae ee a apenas ey Zee 
36 36 36° 367° (367! 36 
5 4 3 2 1 
ae Eg? 1p 6 1 
36 367 367 367 36 
329 
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var(X) = E(X?)—E(x)? 
B20: 2s 
SS ge 
385 
7 BGs 
1]2]3/4]5]6 
1]0]1/2/3)/4]5 
Feo a 23 (4 
3/2/1/o/1/2/3 
ra ae On ee 
54/3/2/1/o/1 
6/5/4/3/2/1/0 
































= DO PS =i)P(2=f)+ SY) P(S =A)P(S2= 


t<j,j—i=0 i>j,i—j=0 


36 


S> Pi =H)P(S2=7)+ So P(S, =4)P(S2 = J) 
i<j,j-i=1 i>ji—j=1 
10 
36 
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P(X=4) = SO P(S=i)P(S2=5)+ SYS) P(S1 =1)P(2 =5) 
i<j,j-i=4 i>ji-j=4 
_ 4 
=" ¢86 
P(X =5) = P(S, = 1)P(S2 = 3) + P(S; = 1)P(S2 = j) 
i<j,j—t=5 i>j,i—J=5 
_ 2 
36 
ou = E(S;) _ E(S2) = 0 since E(S;) = E(S2). 
P(X=2) = P(S,=1,%=1) 


Pq 


P(X = 7) = P(S; =1, So = 6) + P(S; = 2, So = 5) + P(S1 = 3,5, = 4) 
+P(S; = 4,5, = 3) + P(S; = 5,59 = 2) + P(S; = 6,5) = 1) 
= pide + pods + p3g4 + pags + P5g2 + ped 








P(X =12) = P(S; =6,S) =6) 





= P6d6 
VP(X = 2)P(X = 12) = /piqipege 
= v (p196) (pe) 
< (pige) + (pequ) apply the arithmetic/geometric 
=F 2 mean inequality 


< pide + +Peq1 
< Pigs + p2gs + p3qs 
+ pag3 + P5q2 + Pegi 
= P(X =7) 
Thus X cannot be uniformly distributed on the set {2,3,...,12}. Other- 
wise, the above inequality is an equality. 








CHAPTER 4 


Probability generating functions 


4.1. Generating functions 


EXERCISE (4.4). If uo, u1,... has generating function U(s) and vo, v1,... 
has generating function V(s), find V(s) in terms of U(s) when (a) up = 2Un, 
(b) Ont PLC) Oe = hs. 


SOLUTION. (a) The sequence given by 


Un = 2us 
has generating function 
Vis) = S > (2tin)s” a2 SS t,s = 20 (8) (U(s)= Ss" Uns” by the hypothesis) 


(b) The sequence given by 


Un = Un tl 


has generating function 


(oe) 


V(s) = So (un + 1s” 





(U(s) = SS Uns" by the hypothesis) 


n=0 
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(c) The sequence given by 


Un = NUn 
has generating function 
[oe) co co 
V(s) = So (ntin)s” = De NUns” = s(S Uns”)! = sU"(s) 
n=0 n=0 n=0 


EXERCISE (4.5). Let 0 < p=1-—q< 1. Of which sequence is U(s) = 


1/.\/1 — 4pqs? the generating function? 


SOLUTION. Prove that 


a+aje= >> (2) foraéeR 


where 





Define f(x) = (1+ x)*. Then 


f(x) =a(a—1)---(a—k+1)\(1+2)** 


for k = 1,2,.... This implies that 


f®(0) = a(a—1)---(a—k +1) 


By the Maclautin expansion of f(a), we have 





SO (kb) hs 
fee) = ak = (Pat () 


k=0 k=0 


Apply (1) with 2 = —4pqs?, a = —1/2. Then 
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k=0 
oo 1 3 By. fi 2k 
= ( 5)( 5)( 2) ( 2 di 1yPa* "gs?" 
k=0 . 
= 1-3-5---(2k—1 
k=0 . 
LS hen (Qk — 1 
= S- ) ok gk gh 2h 
k=0 : 
bee hek S41 
k=0 st 
cc eee ee eran ey ee | 
=o - 19. 4.6--+(2k)|peqhs* 
mt Ik! 
=. 1-2-3-4-5-6---(2k—1)-(2k) pp op 
ap klk! ae 
k=0 
S (2k 
2ya( ee 
k=0 


Thus 
U(s) = 1//1 — 4pqs? 


is the generating function of the sequence ug, = pq", U2K+1 = 0. 
Note. Since U(s) is an even function, so its Taylor’s expansion consits 
of the power of even degree only. Hence woz441 = 0. 
Remark. If U(s) = ./1—4pqs?, then apply (1) with « = —4pqs?, 
a= 1/2. Then 
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(1—4pqs?)/?_ = = (—4pqs”)* 





_ a) ( 1)F aki gk 5? 





= _11-3-5---(2k—3 
= Se 1)* 1 ( )( 1a p g's 




















k=0 
“ 1-3-5+--(2k-3 
k=0 
— 1 1-3-5-++(2k—3)(2k-1 
-_ S> Gk 1) 2 Cr )( Jig 4-6-+- (2k) |p qs? 
= Ik! 
_< 1 (20s SA he Gees OR S(O 1) 9 9k) % poe 
2 k= 1) klk ie 
k=0 
ao es ey kg 2h 
2+" (ak-1)\k 
Thus 
U(s) = V1 — 4pqs? 
is the generating function of the sequence uaz = SF or ) p'q®, ary = 0 


4.2. Integer-valued random variables 


EXAMPLE (4.14). Let X be a random variable having the Bernoulli 
distribution with parameter p. Then 


P(X =0)=4, P(X = 1) =p, 
where p+q=1, and X has probability generating functio 
Gx(s) px (0)s° + px(1)s" 
= qt+ps. 


EXAMPLE (4.15). Let X be a random variable having the binomial 
distribution with parameters n and p. Then 


PxXS=k) = ee fork = 0,1,2,...,n, 


4.2. INTEGER-VALUED RANDOM VARIABLES 115 


and X has probability generating function 


EXAMPLE (4.16). Let X be a random variable having the Poisson dis- 
tribution with parameters A. Then 


1 
P(X =k) = Me™ for k= 0,1,2,.2., 


and X has probability generating function 


1 
Gx(s) = S- ae 


k! 
k=0 
n 
1 
=e Ars) 
k=0 
“1 
—r A : 5 k 
Sere (use the Maclaurin series Ne a? = e*) 
k=0 
= er(s-), 


EXAMPLE (4.17). Let X be a random variable having the negative 
binomial distribution with parameters n and p. Then 


k-1 
P(X =) =( _ )ora" fork=n,n+1,n+2,..., 


and X has probability generating function 
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a ea (n—1)! n_k—-n_k 
2 Ge mie t 
_ wa n(ntl)--(K-1) in kone 
d. (k—n)! 
a > iyi Won a ba 1) rgi ght (put k= n+ 9 for 7 = 0,1,2,.. 
j=0 . 
= (ps)" ai ii n)(—n ees n-jt digs 
j=0 - 
= (ps)" )0(-1)' & (as)? 
j=0 J 
= (ps)" 5° oS (—qs)’ 
j=0 \ I 
= (ps)"(1— qs)" 
ps n 
= (rea. 


EXERCISE (4.18). If X is a random variable with probability generating 
function Gx(s), and k is a positive integer, show that Y = kX and Z = 
X +k have probability generating functions 


Gy(s) = Gx(s*), Gz(s) = s*Gx(s). 


SOLUTION. Y = kX, for i€ ImY then i = kj for 7 € {0,1,2,...}. 


-) 
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Z=X +k, fori€cImZ theni=j+k for j € {0,1,2,...} 


Gz(s) 




















- si P(Y = kj)s"? 
j=0 

= S PRX = ky) 5H! 
j=0 

= DPX = Jj) 
j=0 

= SUP(X = j)(s* 
j=0 

= Gx(s*) 


So P(Z =j +k)s* 














P(X +k=j+k)s! 
j=0 
k\ > P(X = 7)s! 
j=0 
s*Gx(s) 
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EXERCISE (4.19). If X is uniformly distributed on {0,1,2,...,a}, in 


that 


(tories CTD nc35.a,) 


1 


BOSSE aoe 





show that X has probability generating function 


Gx(s) = 


(a 


Le gatl 


(a+1)-s) 


118 4. PROBABILITY GENERATING FUNCTIONS 


SOLUTION. We have that 


EXERCISE (4.30) 











Gx(s) = S>P(X =k)s* 
k=0 
2 
FS aaa 
jet 
1 a 
a+ rae 
1 — gatl 
7 a+l l—-s 


4.3. Moments 


. Use the method of generating functions to show that 


a random variable having the Poisson distribution, parameter A, has both 
mean and variance equal to A. 


SOLUTION. Let X have the Poisson distribution with parameter X. It 
has probability generating function Gx(s) = e@-)). Hence 











EX?) 
var (X ) 





EXERCISE (4.31) 


Gy (1) = AYXT-Y = d, 
COG Oy ee ey 
E(X?) — E(X)? = (OW? +A) -— 2 =), 


























. If X has the negative binomial distribution with pa- 


rameters n and p, show that 











E(X) = n/p, var(X) = nq/p*, 





where g=1-—p. 


SOLUTION. Let X have the negative binomial distribution with param- 
eters n and p. It has probability generating function Gx(s) = [ps/(1—qs)]|” 


for |s| <q"! 


where gq = 1 — p. Hence 
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op nen ia OSE gy pM ns) pas 
PS \n-1P(1 — 4s) + Gps 
eae (1 — qs)? 











1 fe) — mp — Lips) 
G(s) = np (1 = gqs)2nt2 ; 
This implies 


(mn — 1)p?” + (n + 1)p??“"(1 — p) 
















































































Gx (1) = np ponte 
_ n(n —1)p?rtt + n(n + 1)p?"(1 — p) 
> pent2 
_ p?"(—2np +n? +n) 
— pene 
n? +n —2np 
= 2 ; 
Hence 
non ear e (pomp _ in 
E(X) fa x (1) aaa — grt —. prt = p’ 
2 
; n’+n—-2np n 
E(X*) = Gk (1) + Gk (1) = 5 ee 
Pp Pp 
nmt+n—-2np nn? 
var(X) = E(X?) —E(X)? = + 
(X) = E(X“) — E(X) e a op 
= n — 2np+np _ na np _ n(1 — p) _ ng 
p p p p 


EXERCISE (4.32). Let X be a random variable taking values in the 
finite set {1,2,...,N}. The Dirichlet probability generating function of X 
is defined as the function A(s) = E(X~*). Express the mean of X in terms 
of A. Similarly, express the mean of log X in terms of A. You may find it 
useful to recall that («¥ — 1)/y > logax as y > 0. 
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SOLUTION. The mean of X in terms of A, where A is defined as the 
function A(s) = E(X~*), can be easily found by calculatingA at the point 
s =—1l. To see this when s = —1, 




















A(-1) = E(X~CY) = K(X). 
































If A is defined as the function A(s) = E(s~*), we have that 


so that 


When s = 1, 














It follows that 

















If A is defined as the function A(s) = E(X~*), the mean of log X in 
terms of A can be calculated by using the following argument: 











4.3. MOMENTS 



























































r=1 
N y_] 

= S- (lim ——)P(X = 2) 
REA y—0 y 

Jet 2YP(X = 2) — P(X =2) 

7 aa y—0 y 

a sre 2YP(X = 2) — P(X =2) 

7 aa y0 y 

esis DraileYP(X = x) — P(X =2)| 
y0 y 

= ti eae P(X = 2) — OL P(X = 2) 
y0 y 

= lim a= 
y0 y 


Alternateiely, we have that 














N 
A(s) = E(X~*) = S¢ a °P(X = 2). 
r=1 
It follows that 
N 
A'(s) = Ss" —a2* log zP(X =z) 
z=1 


When s = 0, 














N 
A’(0) = — 5 ~ log aP(X = x) = —E(log X) 
e=1 


so that 
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= lim by the definition of derivative of A at s =0 
yo 
(since A(0)=E(1)=1) 
= lim Aye 4 
y—0 y 


Note that when s = 1, 




















N 
A'(1)=->> 0ST pry = 2) = -K( 

















) 
3 2 xX 
so that 
log X dA 
E = —A/(1) = — — ; 
( X ) (1) ha er 


4.4. Sums of independent random variables 


EXERCISE (4.40). Use Theorem 4.33 to show that the sum of two inde- 
pendent random variables, having the Poisson distribution with parameters 
A and ps respectively, has the Poisson distribution also, with parameter A+. 
Compare your solution to that of Exercise 3.29. 


SOLUTION. Let X and Y be independent random variables with Poisson 
distributions having parameters \ and wp respectively. That is to say, 


Gx(s) = een! 


and 


Gy(s) = eh-D, 
Hence, if Z = X + Y, we have 


Gz(s) = Gx+y(s) 
(by Theorem 4.33) = Gx(s)Gy(s) 
_ er(s-)) eH(s-)) 


= e+n)(s-0), 
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By Uniqueness theorem for probability generating functions, Theorem 4.13, 
Z has the Poisson distribution with parameter + j. 


EXERCISE (4.41). Use generating functions to find the distribution of 
X +Y, where X and Y are independent random variables, X having the 
binomial distribution with parameters m and p, and Y having the bino- 
mial distribution with parameters n and p. Deduce that the sum of n 
independent random variables, each having the Bernoulli distribution with 
parameter p, has the binomial distribution with parameters n and p. 


SOLUTION. Let X and Y be independent random variables with bino- 
mial distributions having parameters m, p and n, p, respectively. That is 
to say, 


Gx(s) = (q+ps)” 
and 


Gy(s) = (q+ ps)”. 
Hence, if Z = X + Y, we have 


Gz(s) = Gx+y(s) 
(by Theorem 4.33) = Gx(s)Gy(s) 
= (q+ ps)'""(q + ps)” 
= (q+ps)™*". 


By Uniqueness theorem for probability generating functions, Theorem 4.13, 
Z has the binomial distribution with parameter m+n and p. 

Let X; for i € {1,2,...n} be be independent random variables with 
Bernoulli distributions having parameter p. That is to say, 


Gx, = (q+ ps) 
Hence, if S = X; + Xo +---+ Xn, we have 


Gz (8) = Gx,4X24--4+Xn(8) 
(by Theorem 4.33) = II G‘x,(s) 
i=1 


= (q+ ps)”. 
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By Uniqueness theorem for probability generating functions, Theorem 4.13, 
S has the binomial distribution with parameters n and p. 


EXERCISE (4.42). Each egg laid by a hen falls onto the concrete floor 
of the henhouse and cracks with probability p. If the number of eggs laid 
today by the hen has the Poisson distribution, parameter A, use generat- 
ing functions to show that the number of uncracked eggs has the Poisson 
distribution with parameter A(1 — p). 


SOLUTION. Let A; be the event that the ith egg is laid today by the 
hen. For i = 1,2,...,n, we define the discrete random variable X; by 


1 if the 7th egg is uncracked, 
0 if the ith egg is cracked. 


X;(A;) = 


Each X; takes values in {0,1} and has mass function given by 


and 


P(X,;=1)=1-p. 


Hence, each X; has the Bernoulli distribution with parameter 1—p. Let N 
be the number number of eggs laid today by the hen, and S' be the number 
of uncracked eggs. Then 


N 
oS S> xy (Nis a discrete random variable) 
i=l 


Since N and X; are independent random variables, we conclude from 
the random sum formula (4.37) that the tota; number S has probability 
generating function 


G(s) = Gn(Gx,(s)) = Gn(p+ (1—p)s) since Gx,(s) = p+ (1 — p)s 
By our hypothesis then 


Gn(s) = ra ame) 
this implies that 
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G(s) = eP+C—p)s-1] — Al-p)s—(—-p)] — eAC-p)(s-1) 


By Uniqueness theorem for probability generating functions, Theorem 4.13, 
S has the Poisson distribution with parameter \(1— p). 


4.5. Problems 


PROBLEM (4.5.1). Let X have probability generating function Gx(s) 
and let un = P(X > n). Show that the generating function U(s) of the 
sequence ug, U1,... satisfies 


(1 — s)U(s) = 1— Gx/(s), 
whenever the series defining these generating functions converge. 


SOLUTION. We have that 
(1 — s)U(s) = U(s) — sU(s) 
= d us" = > ust! 
= uo + Yt — Uj 1) 


= Hob DIP (X >i+1)— P(X > i)]s* 


(since P(X >1+1)—- P(X s ie = —P(X =1)) 


(since Gx(s) = P(X = 0) + 02, P(X = i)s’) 














whenever the series defining these generating functions converge. 
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PROBLEM (4.5.2). A symmetrical die is thrown independently seven 
times. What is the probability that the total number of points obtained is 
14? (Oxford 1974M) 


PROBLEM. Consider 


ttre t+ag+44+%5 +46 = 14 
such that 1 < 2; < 6 for every 1 <i<6. 
First, let y; = 2; — 1 for 1 <i < 6. We will count integer solutions of 
the equation 
yi+tyot+y3+yatys tye = 8 with O < y < 5, , as there is a 
straightforward bijection between such solutions and the solutions of the 
original equation. There are 


S6—L)\. fis 
6-1 / \6 
non-negative solutions to this equation, when we ignore the upper bounds. 


Let A; be the set of solutions with y; > 6. Then we are interested in Ga) = 
|A; U Ag U A3 U Ag U As U Ao]. Applying inclusion-exclusion, we have 


6 
|A1 U Ap U A3U Ag U As U Agl = 5° | Ail - Ss" |AiN A;| 
i=1 1<i<j<6 
Let x; be the number of points obtained in 7th throw for 7 = 1,2,..., 7. 
Then the problem reduces to finding the number of distinct positive integer- 
valued vectors (41, %2,...,27) such that 


ty +tg+azt+agt+a5+a%¢+ 27 = 14, 


where 1 < 2; < 6. Let y, = x7; —1 for 1 <2 < 7. We will count integer 
solutions of the equation 


yityotyst+yat+ystyety7=7 
where 0 < y; < 6, as there is a straightforward bijection between such 
solutions and the solutions of the original equation. There are 


(3779- (2) 


non-negative solutions to this equation, when we ignore the upper bounds. 
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If y; = 7 for some 7, then y; = 0 for 7 #7. In this case, there are 7 
solutions. 

If y; = 6 for some 7, then there exists some j #7 such that y; = 1 and 
ye = 0 fork Ati and k # 7. In this case, there are 7-6 = 42 solutions. Thus 
the number of solutions satisfying 1 < x; < 6 for all 7 of the given equation 
is (@) — 49. 

The probability that the total number of points obtained be 14 is that 


ar[(G) — 49]. 


PROBLEM (4.5.3). Three players, Alan, Bob, and Cindy, throw a per- 
fect die in turn independently in the orderA, B,C, A,... until one wins by 
throwing a 5 or a 6. Show that the probability generating function F’(s) 
for the random variable X which takes the value r if the game ends on the 
rth throw can be written as 


9s 6s? 4s 


F — | i 
(s) 27 — 883 





' 27-853 | 27 — 853° 


Hence find the probabilities of winning for Alan, Bob, and Cindy. Find the 
mean duration of the game. (Oxford 1973M) 


SOLUTION. Clearly, X has the geometric distribution with parameter 
p = 2/6 and 





MX == (GY 1=@Ey7 


for r= 1,2,3.... Hence the pgf of X is then 
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F(s) = 3 s*P(X =k) 

k=0 

= yr o(ayGye (since P(X = 0) 
k=1 

= (5) DG) 

k=1 

doe? 

= 3G) 

ee 
21- 38 

5g 


The probability of winning for Alan, 


Pa =S P(X = 3k +1) 


> 
j=) 


II 
Me 
IRS 
Se 
cd 
+ 
i 


> 
ll 
° 


| 

live 
WE 
| % 


ll 
° 
ae 





| 
Blo ele ae 
; mR 
| 
When 


The probability of winning for Bob, 
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“—— X = 3k +2) 


T 
> 
ee 
ll 
Se 
> 
+ 
iw) 


j=) 


OlDN wWwlrR 
[seas Se Dera 
a 





19° 
Since X has probability generating function F'(s) = s/(3 — 2s), it follows 
that 











E(X) = F"(1) 














PROBLEM (4.5.4). A player undertakes trials, and the probability of 
success at each trial is p. A turn consists of a sequence of trials up to 
the first failure. Obtain the probability generating function for the total 
number of successes in N turns. Show that the mean of this distribution is 
Np(1—p)~} and find its variance. (Oxford 1974M) 


SOLUTION. For i= 1,2,...,N, let X; be the discrete random varaible 
that represent the number of successes in the 7th turn. 

It is easy to see that random variables X; are independent and have 
identical distributions. Each X; has mass function given by 


(tor he 05 122325) P(X; =k) =p*q 
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and each X; has probability generating function 


Gx,(s) = 5° s*P(X; =k) 


Q 


~ T—ps’ 
It follows that the sum Sy = X, + Xo +---+ Xy of N independent ran- 


dom variables, each taking values in {0,1,2,...}, has probability generating 
function given by 


Gu (8) = Gx (s)Gra(8) ++ @xee(8) = [Gx (8) = (EL ®, 


Hence 
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E(Sh) = GS, (1) + Gy (1) 

_d Npq’ 2 Np 
ds (l—ps)"™| 29 1p 

= Npg(EN =T)Gp)| a Np 

(haps)? ay Lap 

_ N(N+1)p?q™ | Np 

= gh+2 Of p 
N(N+1)p? | Np 








(l—p)? ° 1=p’ 


var(X) = E(S%,) — E(Sw)? 

N(N+1)p? Np N?p? 
(Lap)? top “(=p 
_ N?p? + Np? + Np(1 — p) — N?p? 

7 (ip)? 











Np(p + q) 
q? 
Np 
(Lp)? 





PROBLEM (4.5.5). Each year a tree of a particular type flowers once, 
and the probability that it has n flowers is (1—p)p", n = 0,1,2,..., where 
0 <p <1. Each flower has probability 5 of producing a ripe fruit, inde- 
pendently of all other flowers. Find the probability that in a given year 

(a) the tree produces r ripe fruits, 

(b) the tree had n flowers, given that it produces r ripe fruits. 
(Oxford 1982M) 


SOLUTION. (a) The number N of flowers has probability generating 
function 
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G(s) = 5° s*P(X =k) 
k=0 

= > (1 — p)p*s* 
k=0 


II 
‘iP 
= 
4 
= 

> 





Let X; be the random variable that takes value 1 if the ithe flower pro- 
duce a ripe fruit, and takes value 0 otherwise. Each X; has the Bernoulli 
distribution with parameter 5 and consequently each X; has probability 
generating function 


Since N and_X; are independent, we conclude from the random sum formula 
(4.37) that the total number S = Xo + X1 + Xo +---+ Xy of flowers 
producing ripe fruits has generating function 
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Gs(s) = Gn(Gx,(s)) 
eee ae 
1 — p(s +48) 
2(1 — p) 
(2—p) —ps 
2(1—p) 








giving 


the coefficient in the sum above with respect to s”. 


(b) Let A be the event that the tree has n flowers, and let B be the 
event that the tree produces r ripe fruits, 


P(A| B)= P(ANB) 
P(B) 
- p(B| ALA 
P(B) 
The probability that the tree produces r ripe fruits, given that it has n 
flowers, is 


rel ay=(")OrG 


ee 


Thus 


134 4. PROBABILITY GENERATING FUNCTIONS 


n = nm nm) (1yn(q n(9 _ m\r+l 
pal B)= ("\Gyr gr Gee _ (GG pp"(2- p) 


(2—p)rtt 


=(*)rranre-n 


r 








PROBLEM (4.5.6). An unfair coin is tossed n times, each outcome is 
independent of all the others, and on each toss a head is shown with prob- 
ability p. The total number of heads shown is X. Use the probability 
generating function of X to find 

(a) the mean and variance of X, 

(b) the probability that X is even, 

(c) the probability that X is divisible by 3. 

(Oxford 1980M) 


SOLUTION. (a) The number X of heads after n tosses has the binomial 
distribution with parameters n and p, and consequently X has probability 
generating function 





Gx(s) = E(s*) 
= (q+ ps)". 











where g = 1 — p. Hence 











E(X) = Gx (1) = np, 

E(X?) = Gk(1) + Gx (1) = npg + np, 
var X = E(X*) — E(X)? = npaq, 

in agreement with the calculations of Exercise 2.37. 


(b) Solution 1. For k = 0,1,2,..., Ee we have that 









































so that 
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Solution 2. We have 


Gx(s) =pot+pist+ pos? +> 
Observe that 


Gx(1)=potpitpet::: 
Gx(-1) = po —pitpe--:: 
Thus 


Gx (1) + Gx(-1) = 2po + 2p2 + 2pg +--: 
so that 


(00 + pa pat) = 5IGx(1) + Gx(-D) 
= Sl -) +0)" + (0 -) + (-} 
= 5 [1+ (1 — 2p)" 


(c) Solution 1. For k = 0,1,2,..., ||, we have that 


so that 


Solution 2. We have 


Gx(s) =po+pis+ pos? +:::. 


Let w and w? be the primitive complex cube roots of unity. Thus There are 


3 complex cube roots of unity: 1, w, w?. 
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Gx(1)=potprt+pet+-::: 
Gx (w) = po + piw + pow? +++ 
Gx (w?) = po + piw? + pow? +--- 
Thus 


Gx (1) + Gx (w) + Gx (w?) = (po + pi + p2 ++++) + (po + piw + paw? +++) + (po + piw? + pow? +--+ 


= 3potpi(ltwtw)+po(l+u7 +w*) + pa(l+w? +w®) ++: 

















= 3po + pi(Lt+w+w?)+po(ltw+w*) +p3(lt141)t-:: 


= 3po + pi -0+p2-0+3p3+--- 





so that 


(00 + Ps +96 +--+) = Z1Gx(1) + Gx(u) + @x(u?) 


= {la —p) +p)" + [(1—p) + pw] [(1 — p) + pw”]"} 


PROBLEM (4.5.7). Let X and Y be independent random variables hav- 
ing Poisson distributions with parameters A and j4, respectively. Prove that 
X +Y has a Poisson distribution and that var(X + Y) = var(X) + var(Y). 
Find the conditional probability PX =k|X+Y =n) forO<k<n, and 
hence show that the conditional expectation of X given that X + Y =n, 
that is, 

















B(X | X+Y¥=n)=S okP(X =k|X+Y =n), 
k=0 


is nA/(A + ps). (Oxford 1983M) 


SOLUTION. Let X and Y be independent random variables with Poisson 
distributions having parameters A and py, respectively. That is to say, 


Gx(s) = oe} 


and 


Gy(s) = eh-D, 
Hence, if Z = X + Y, we have 
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Gz(s) = Gx+y(s) 
(by Theorem 4.33) = Gx(s)Gy(s) 
= eX(s-1) eu(s-1) 
= eAte)(s-1). 
By Uniqueness theorem for probability generating functions, Theorem 4.13, 


Z=X+Y has the Poisson distribution with parameter A + w. 
We have 








var(X +Y)=E ae Y)?] — [E(X + Y)/? 
= E(X? + 2XY + Y7) - [E(X) + E(Y)P 
= E(X’) + 2E(XY) + E(Y*) — [E(X)P — 2E(X)E(Y) — [E(Y)? 
(since X and Y are pidependens so E(XY) = E(X)E(Y)) 
= {E(X*) — [E(X)]’} + {E(¥”) — [EY))} 
= var(X) + var(Y) 





















































For k = 0,1,2,...,n, we have that 




















7 _ oy (X= bY =n-h) 
Epo ate ein lor = = aera =a) 
(since X and Y are independent) = P(X +Y =n) 


(since X + Y has Poisson distribution with parameter + j) 
Align te 
EA + pyre“ OF 
pi Perse heh 
~ k(n — KIA + we OHH) 


- (acu 











For k = 0,1,2,...,n, we have that 
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R(X =k|X+Y¥ =n) =) kP(X =k|X+Y =n) 
k=0 


(ao 


k=0 


1 = n 
= oar Dl aur 
(A+ pu)” <<" \k 


Use Binomial Theorem for (\ + j)" 


(A+ 4)" = 3 Cae 


Take the derivative in the variable , we have 


n(r + p)” t= 3a, pee 


Multiply both sides by A, we have 


MO) SK (;) kame 
0 


Thus 























PROBLEM 2 (*4.5.8). A fair coin is tossed a random number N of times, 
giving a total of X heads and Y tails. You showed in Problem 3.6.14 that 
X and Y are independent if N has the Poisson distribution. Use generating 
functions to show that the converse is valid too: if X and Y are independent 
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and the generating function Gy(s) of N is assumed to exist for values of s 
in a neighbourhood of s = 1, then N has the Poisson distribution. 


SOLUTION. Since the coin is fair, so the probability of getting head is 
p=1/2. For = 1.2....,.N, we define the discrete random variable X; by 


xy 1 if the 7th toss is H, 
‘ )0. if the ith toss is T. 


Each X; has the Bernoulli distribution with parameter p = 1/2 and the 
probability generating function 


1 1 
Gx, (s) =qt+ps = 3 + ran 


By Random sum formula, Theorem 4.36, 


Similarly, 


Ge\= Gx (5 + 5°) 


Since X and Y are independent, so X and N — X are independent (since 
Y=N-X). 


Gyn(s) = Gxty(s) 


(since X and Y are independent) = Gx(s)Gy(s) 
(since Gx(s) = Gy(s)) = [Gx(s)} 
=[n(5 +592 


Let Gy(s) = G(1—s). Then Gy(s) = G(1— s). Thus 
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Gn +59) = @(L- 5-58) 
= (ames 





But since Gy(s) = [Gy(5 + 55)|?, so G(1 — s) = [G(+5*)]?. Thus the 
function G has the property 


aH =(eQr a) 
Use (1) n times, we obtain 
Gt) = 1GG)P = GP =-- =e" 


Use the Maclaurin expansion, we have 


Gn(1—s) = Gy(1) — Gy (1) + 0(s?) 
It implies that 


G(s) = Gy (1) — Gy(1)s + 0o(s”) 
= 1— ps + o(s”) 
where p = Gy (1) = E(N). Thus 

















Thus 


Gy(s) =G(1-s)= e Ms) — eu(s-)) 
By Uniqueness theorem for probability generating functions, Theorem 4.13, 
N has the Poisson distribution with parameter pu. 


4.5. PROBLEMS 141 


PROBLEM 38 (4.5.9). Coupon-collecting problem. Each packet of a cer- 
tain breakfast cereal contains one token, coloured either red, blue, or green. 
The coloured tokens are distributed randomly among the packets, each 
colour being equally likely. Let X be the random variable which takes the 
value 7 when I find my first red token in the jth packet which I open. Obtain 
the probability generating function of X, and hence find its expectation. 
More generally, suppose that there are tokens of m different colours, all 
equally likely. Let Y be the random variable which takes the value 7 when I 
first obtain a full set, of at least one token of each colour, when I open my jth 
packet. Find the generating function of Y, and show that its expectation 
is m(1+5+4+---+4). (Oxford 1985M) 





SOLUTION. The the random variable X has the geometric distribution 
with parameter p = 4, and consequently X has probability generating 





ae 
function 
ps 
GC = 
x(8) 1—(1-p)s 
zs 
1—(1-)s 
_ 8s 
— 3-28 
Hence 














For 7 = 0,1,2,...,m— 1, let Y; be the additional number of tokens 
collected, after obtaining 7 distinct colours, before a new colour is collected. 
Thus each Y; has the geometric distribution with parameter (m — i)/m 
having probability generating function 
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for 7 = 0,1,2,...,m—1. Thus 


X= Yo Vio a ok Yani 


has probability generating function 


Gy (8) = Gyo(s)Gy (s)Gy2(s) --- Gy; (5) 


_(m—0)s (m—1)s (m—2)s — [m—(m—1)]s° 








m-0s m-l1s m-—2s m—(m-—1) 
og Gratis t= 2)8 S 
~ 9's | m—2s m—(m=—1)s 

7 (m—i)s 

IT m — 1s 


We have that 








ar — 18s 
m-1 : 
d (m—1)s m—j)s 
-y ee) T= a 
7=0 x J 
m-1 





au ee 
: 7 


or m— js 


so that 
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mm =i) yy (m= 3)s 
; _ = 
1)= 
y (1) lin way Laie 
i=0 j#t s<l1 
m-1 2 
= ae) da 
= — — 
=o. ‘) gee 
m—1 a m—j 
7 fom | Eee 
i=0 j#t 
m—-1 
_ m 
— Lami 
1=0 
m—-1 1 
ge eer 
1=0 
1 1 1 
=mlt+s+5+--4 
aan ae 
Thus E(Y) = GY(1) =m(1+h4+i4+---+24) 














PROBLEM (4.5.10). Define the mean value of a discrete random variable 

and the probability generating function ¢. Show that the mean value is 
¢'(1). If @(s) has the form p(s)/q(s) show that the mean value is (p’(1) — 
d())/a(). 
Two duellists, A and B, fire at each other in turn until one hits the other. 
Each duellist has the same probability of obtaining a hit with each shot 
fired, these probabilities being a for A and 6 for B. If A fires the first shot, 
calculate the probability that A wins the duel. Find also the probability 
distribution of the number of shots fired before the duel terminates. What 
is the expected number of shots fired? (Oxford 1976M) 


SOLUTION. If X is a discrete random variable, the expectation of X 
is denoted by E(X) and defined by 


























EXy= Sars) 
z€Im X 
whenever this sum converges absolutely, in that }°.,|2P(X = x)| < oo. 
The probability generating function (or pgf) of X is the function 
@(s) defined by 
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o(s) =pot+pis+ ps? +-°-, 


for all values of s for which the right-hand side converges absolutely. 
Thus 





k=1 
so that 
$'(1) = 50 kk 'P(X = k) 
R=] 
= ys xP(X =x) 
xveIm X 





where Im X = {0,1,2,...}. Thus ¢’(1) = E(X) as required. 
Let X has probability generating function ¢(s) of the form p(s)/q(s). 
Then 











so that p(1) = q(1). Hence 
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(since p(1) = q(1)) = 











Thus E(X) = (p'(1) — q'(1))/¢(1). 
Let X be the random variable that takes value k at the kth shot and 
A wins the duel. It is easy to see that 





P(A wins) = doer =2k+1 


= . =a)" (1—b)Fa 


k=0 


= draft = aka — ay 


= a 
~ 1-(1-a)(1—-5) 
= 
a+tb—ab 





Let Y be the random variable that takes value k when the duel termi- 
nate at kth shot. Thus 


(l1—a)*(1—b)*-1b ifn=2k 
(l—a)FK(1—b)*a ifn =2k4+1 


for k = 0,1,2,.... We have that 
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Gy(s) = 5 >(1— a)¥(1 — b)*1bs7* + $01 — a) *(1 — 6) 
k=1 k=0 
é —_ (1 = a)(1 — 6)Ks?* + as (1 — a)F(1 — yo? 
k=1 k=0 
= —_ (1 —a)(1—B)s?]* +s Sl —a)(1— 8) s*] 
k=1 k=0 
6b = (l-a)— b)s? 1 





“T= (1—a)(1 — b)s? 





b(1 — a)s? +48 





so that 














E(Y) = Gy(1) = 








PROBLEM (4.5.11). There is a random number N of foreign objects 
in my soup, with mean yp and finite variance. Each object is a fly with 
probability p, and otherwise a spider; different objects have independent 
types. Let F’ be the number of flies and S the number of spiders. 

(a) Show that Gr(s) = Gx(ps+1-—p). [You should present a clear 
statement of any general result used.| 

(b) Suppose N has the Poisson distribution with parameter pp. Show 
that F’ has the Poisson distribution with parameter yp, and that F' and S$ 
are independent. 

(c) Let p = 5 and suppose F' and S are independent. [You are given 
nothing about the distribution of N.] Show that Gy(s) = Gn(3[1 + 5])?. 
By working with the function H(s) = Gy(1—s) or otherwise, deduce that 
N has a Poisson distribution. 

You may assume that [1 + (2/n) + o(n—!)]”" > e® as n + oo. (Cambridge 
2002) 


SOLUTION. (a) For = 1.2...., N, we define the discrete random variable 
F; by 
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R= 1 if the ith object is a fly, 
‘" )0. if the ith object is a spider. 


Each F; has the Bernoulli distribution with parameter p and the probability 
generating function 


Gr,(s) =ps+q=pst+1—p 
Since N and the F; are independent, we conclude from the random sum 
formula (4.37) that the total number F’ = Fi + Fy +---+ Fy of flies has 
probability generating function 


Gr(s) = Gn(Gr,(s)) 
= Gnu(ps+1—p) 


(b) The number N of of foreign objects in my soup has the Poisson dis- 
tribution with parameter jy, and consequently N has probability generating 
function 


Gy(s) = ebe=l). 


Hence 


Gr(s) = Gn(ps + 1— p) 
= el(ps+1—p—1) 


= ebp(s-1) 


By Uniqueness theorem for probability generating functions, Theorem 4.13, 
F has the Poisson distribution with parameter pp. 
For = 1.2....,N, we define the discrete random variable S; by 


$= 1 if the zth object is a spider, 
‘" )0. if the ith object is a fly. 


Each S$; has the Bernoulli distribution with parameter 1 — p and the prob- 
ability generating function 


Gs,(s) =(1—p)s +p. 
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Since N and the S; are independent, we conclude from the random sum 
formula (4.37) that the total number S = S$; + Sz +---+ Sy of flies has 
probability generating function 


Gr(s) = Gn(Gs, (s)) 
= Gn((1—p)s +p) 
— eH[(l—-p)stp—]] 
= eH(-p)(s-1) 
By Uniqueness theorem for probability generating functions, Theorem 4.13, 


S has the Poisson distribution with parameter ju(1 — p). 
We have 








P(F=k,S=l) =P(F=k,N=k41) 
. k| N=k+)P(N =k4+1) 























1 
a l k+l —p 
SE aaah 
= a i A Less 
kill "k= p) (k+oI © 
1 
mie CoP ee wees 


=F lupyhe#? F [u(t — pte?) 
=P(F =hyP(S =). 
Hence F' and S are independent. 
(c) Let p = §. Since Gr(s) = Gy(ps +1 —p) and Gs(s) = Gy([1 — 
p|s +p), it follows that 
1 
Gr(s) = Gy (Zl +5) 


and 


Gs(s) = Gx [1 + s]). 
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Since F and S are independent, so are F' and N — F are independent (since 


S=N-F). 


Gy(s) = Grys(s) 
(since F and S' are independent) = Gp(s)G5(s) 
(since G(s) = Gs(s)) = [Gr(s))? 
={Gn(5[l +s)? 
Let H(s) = Gy(1— ss). Thus 





Cece) ars ; _ 5°) 
a 
- () 


But since G(s) = [Gn(5 + 55)]?, so H(1— 8) = [H(45%)|?. Thus the 
function H has the property 


H()=(AGP 
use (1) n times, we obtain 
H(t) = [HGP = (HGP = = GP 


Use the Maclaurin expansion, we have 


Gy(1—s) = Gy(1) — Giy(1) + o(s’) 
This implies that 


H(s) = Gy(1) — Giy(1)s + o(s”) 
= 1- ps + o(s”) 
where p = Gy (1) = E(N). Thus 
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Sicne (2) holds for any s, so 


H(s) = tim ft. — #8 4 o( Sy] = 


n— 00 2” qn 
Thus 
Gy(s) = H(1—s) = e#C-») = es). By Uniqueness theorem for 
probability generating functions, Theorem 4.13, N has the Poisson distri- 
bution with parameter pu. 


CHAPTER 5 


Distribution functions and density functions 


5.1. Distribution functions 


EXERCISE (5.11). Let X be a random variable taking integer values 
such that P(X = k) = p;, fork =...,—1,0,1,.... Show that the distribu- 
tion function of X satisfies 


Fx (b) — Fx (a) = pati + Pat2 +--+. + Po 
for all integers a, b with a < b. 


SOLUTION. For a < 8, 





Fx (b) — Fx(a) = P(a < X <b) 














=P( LJ (X=) 
k=a+1 
b 
= > WX =h) 
k=a+1 
b 
= De Pk 
k=a+1 


= Pat+1 + Pa+2 + +--+ Po- 


EXERCISE (5.12). If X is a random variable and c is a real number 
such that P(X = c) > 0, show that the distribution function Fy (x) of X 
is discontinuous at the point x = c. Is the converse true? 


SOLUTION. For € > 0, 
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P(X =c) =P({X < c}\{X < c}) 
= P(X <c)—-P(X <c) 
=P(X <c)—limP(X <c—¢) 
= lim[P(X < c) - P(X <e-6)] 


since the event {X < c—} is a subset of the event {X < c}. Hence 


PX = ¢)= lim [Fx (c) — Fx(ce—)]. 


Since P(X = c) > 0, it follows that lim,.,9|F'x(c) — Fx (c— €)] > 0 so that 
F'x(c— €) does not approach to F'x(c) as € f 0. 
For x < c, the event {X < x} is a subset of the event {X < c}. Hence 





P(X <2) <P(X <c 














lI 

J 
Fama. 
ae 
>< 
IA 

~~" iC —" 

“~Y 
yo 
man 
> 
| 
° 
“~Y 
i 


t 
is 


(since P(X =c) > 0) < Fx 


so that 


Fx (x) < Fx(c) 


for any x <c. This implies that 


lim Fx(x) < Fx(c). 
eo 
ZC 


Therefore the distribution function F'x (x) of X is discontinuous at the point 
i= 

The converse is true since suppose that the distribution function Fy (x) 
of X is discontinuous at the point « = c. Then 
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LC 
LC 
Note that lim | — , Fx (a) < Fx(c) since Fx (x) of X is discontinuous at 
LC 


the point «= c. 


EXERCISE (5.13). Express the distribution function of Y = max{0, X} 
in terms of the distribution function Fy of X. 


SOLUTION. For any w € 2, we have 


Y(w) = max(0, X(w)) 
> 0. 


Hence for y < 0, 


so that 


Thus F,(y) = 0 for any y < 0. If y > 0, then 


{Y < y} = max((0,X) < y) 
={X Sut 
so Fy(y) = Fx(y). Hence 
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EXERCISE (5.14). The real number m is called a median of the random 
variable X if 


Show that every random variable has at least one median. 


5.2. Examples of distribution functions 


EXERCISE (5.18). Show that if F; and F» are distribution functions, 
then so is the function F(x) = aFi(x) + (1 — a) F(x) for any a satisfying 
O<a<l. 


SOLUTION. We need to check that F' satisfies (5.5) A non-negative com- 
bination of monotonic non-decreasing functions, is monotonic non-decreasing. 
Hence F' is monotonic non-decreasing. 


We need to check that F' satisfies (5.6). We have that 


line: Fhe) = lim [afi (2) + (1 —a)Fo(z)| 


L—-CO x 
=a lim Fi(e)+(—a) lim F,(2)] 


(since: lim, .o6 45 (@) = 0 for 7 = 1,2) 
=0 


We need to check that F' satisfies (5.7). We have that 


lim F(x) = Jim [oFi (2) + (1-a)Fo(2)| 


=L—Co 


= a lim Fy(xz) + (1—a) lim Fo(2)| 


«LOO 
(since lim, 444 2; (2) = 1 for t= 1,2) 
=a+(l-a) 
=1 


We need to check that F' satisfies (5.8). We have that 
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lim F(a+e)= limlaF (x +e)+(1-—a)Fo(a + )] 
=alimF 1—a)lim F: 
oli i(v +e) + (L-a) im >(a + €)] 


(since F; is continous from the right for 7 = 1, 2) 
= aF\ (x) + (1 — a) Fo(z) 
=H 2). 


Hence F is a distribution function. 


EXERCISE (5.19). Let 
(for ¢ € R.) Ee = cf el du 


For what value of c is F’ a distribution function? 


SOLUTION. We have that 


+00 
fia ee) cf el du 


wL—->CoO oo 


0 +00 
= af el du+ | el dul 
—oo 0 


Sele te |. 
(smce Tina) 6! = lim ee = 0) 

==) 

= 2c 
The function F(x) satisfies (5.5), (5.6), and (5.8), so F(z) is a distribution 
function if and only if limg-... F(x) = 1 2c=1 Cc 5: 








5.3. Continuous random variables 
EXERCISE (5.30). A random variable X has density function 
2x if0<a2<1, 
f(x) = 
0 otherwise. 


Find the distribution function of X. 
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SOLUTION. We have that 


If x < 0, then 


If0<a< 1, then 


—oo 0 

0 ea 
=f odu+ f 2u du 

—oo 0 
=042? 


If x > 1, then 


re=f sedans f senaur f° pean 


0 al x 
=| odu+ f 2udu+ | Odu 
—co 0 1 


=0+1+0 
=1. 


Thus the required distribution function is 
0 ifx<0O, 


PajSst ey sf0<e-< 1, 
1} ese T 
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Note that F(x) increases monotonically from 0 to 1 as is required for a 
distribution function. It should also be noted that F(x) in this case is 
continuous. 


EXERCISE (5.31). If X has density function 


(for x € R,) f(a) = sel 


find the distribution function of X. This is called the bilateral (or double) 
exponential distribution. 


SOLUTION. We have that 


=f fludu 
If x < 0, then 
F(x) = is 56 du 
1 a 
=e 
2 
If x > 0, then 





—oo 0 
0 ry 
= serdut [ ~e “du 
ie 0 2 
Or ie 
es 
=f oF 88 
1 
ee eae 
5e 


Thus the required distribution function is 
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loz : 

ze ifx <0, 
F(x) a ‘4 loj-« : 

Lape ifx> 0. 


Note that F(x) increases monotonically from 0 to 1 as is required for a 
distribution function. It should also be noted that F(x) in this case is 
continuous. 
EXERCISE (5.32). If X has distribution function 
snes! (ey a 
oe pie for —coo<a <0, 


1420? 
Si?) for 0 <4 <a, 


show that X is continuous and find its density function. 


SOLUTION. Let X have distribution function 


1420? 


1 
F(z) = 2(+a2) for —coo< 2a <0, 
3-02) for 0 < x < ©#. 


Since 


lim F(x) = lim F(x) — F(0 
-_ (x) im (x) (0), 


it follows that F' is is continuous at x = 0, and hence for all « and 





F'(2) ” (ey for -o<2<0, 
W422 for 0 <L<K mw. 


Moroever, 


lim Beat) a ee ee) 
x0 x x0 x 


1+ 2x? — (14+ 27) 
1m 
x40 2x(1 + 2?) 





x 
~ 210 2(1 + 22) 
=0 


and 
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ee 
lim pee HU) =e a 
axt0O x xt0O x 
_ 2 
sii I= (1:2) 
x10 2x(1 + x") 
eG ae 
«lO 2(1 + 2?) 
=0 
so that 
F(z) = 
lim Oty =0 
«z—0 x 
and hence F’(0) = 0. 
Let 
f(e)= <0 if ¢ = 0, 
Since f is a non-negative function and F’(a) = f(a) for all x and we 
have 
(by Newton-Leibniz formula) / f (u) du 
(since F(—oo) = limy_,-o F(u) = limy+—co eee =() 
so that 


F(2) = / 7 Fah 


for « € R. This implies that X is a continuous random variable and its 
density function is f(a) be defined by 


a) =< 0 if ¢ = 0, 
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EXERCISE (5.33). Find the distribution function of the so-called ‘ex- 
treme value’ density function 


(for z € R.) f(x) = exp(—z —e *) 


SOLUTION. We have that 





(since d(—e~") = (—e~")/ du = e~“ du) = / e© * d(—e-") 
— ee” : 
=e ° — lim e* 
uU—> — Oo 
(sinte limi 8 0) =e 
(for  € R.) = exp(—e ”) 


5.4. Some common density functions 


EXERCISE (5.45). For what values of its parameters is the gamma dis- 
tribution also an exponential distribution? 


SOLUTION. The gamma distribution also an exponential distribution if 
and only if 


1 


w,w-l —rAr __ —Ax 
ca x = de 


Compare both sides, we have w — 1 so that w = 1. Then 
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T(1) 


lI 
ao 
8 
- 
8 
Q. 
8 


So when w = 1, 


1 


Tw) 


for « > 0. Thus the gamma distribution also an exponential distribution if 
and only if w = 1, arbitrary positive X. 


Negu-t —AX\x = Ne” 


EXERCISE (5.46). Show that the gamma function I'(w) satisfies [(w) = 
(w—1)P'(w—1) for w > 1, and deduce that I'(n) = (n—1)! for n = 1,2,3,... 


SOLUTION. We will use integration by parts with: 


u=cv"—'! and dv =e" dr 


to get 
du = (w — 1)2”~? dx and v = —e™®. 
Then, the integration by parts gives us: 
a=b = 
T(w) = lim [-2~'e~*| ee » | ae de. 


Evaluating at « = b and x = 0 for the first term, and using the definition 
of the gamma function (provided w — 1 > 0) for the second term, we have: 





b> 00 e 


pe-l 
I(w) = — lim ; | +(w—1)P(w-1). 
Since b + oo, we can consider b > 1. Take m € N and m > w — 1, we have 


het ye: 
Thus 
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w-1 pm 
0< — < —. 
eb eb 


Using L’Hospital’s Rule m times, we obtain 


By squeeze theorem, 





Thus 


T(w) = (w— DI (w — 1). 
By the fact that [(w) = (w — 1)['(w — 1), 


I(n) = (n-1DI(n- 1) 
= (n—1)(n — 2)I'(n —- 2) 
= (n — 1)(n — 2)(n — 3)--- (2)(Q)PC) 


And, since by the definition of the gamma function, 





we have 


I'(n) = (n—-1)! 
as was to be proved. 
We can take the exponent and the natural log of the numerator without 


changing the limit. Doing so, we get: 


w—1 —1)] 
lim : = lim eepile yi Then, because both the 
b00 | eb b-00 exp(b) 


numerator and denominator are exponents, we can write the limit as: 
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w—1 
lim * | = jim {exp[(w — 1) Inb — 6}}Manipulating the limit a 
00 


b> 00 e? 


bit more, so that we can easily apply L’H6pital’s Rule, we get: 


jim |=] = — tim {exp [(w - 0 (5? -1)]} 


EXERCISE (5.47). Let 
oe 2 
r= | a ae 
—Co 


By changing variables to polar coordinates, show that 


20 ee) 
Pay) ee? ddy = | | e-” rdr dd, 
R2 0=0 Jr=0 


and deduce that I = \/7. 


SOLUTION. We have that 


eS 2 
i e” dz. 
—oo 


It obviously does not matter what we call the variable, so we also have 


°° 2 
| er Pie 
—oo 


We can now multiply these two expressions together to get 


ea eo? ax | ew dy 


(by Fubini’s theorem) = if , e* —Y dady 


ah eo? —¥" dady. 
R2 


We can rewrite this using polar coordinates, noting that «2? + y? = r? and 


dxdy = rdr dé. We get 
2a oe) ¥ 
pe f a re" drdé 
0=0 Jr=0 


= an f re “a dr. 
r=0 
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We now substitute u = r? 


integral becomes 


, So u also runs from 0 to oo and du = 2rdr. The 


sol = /n. 


f(a) = —Le~* is the probability density functio of the random vari- 


N 


able X having the standard normal distribution. So 


cece a2 
i e 2=1 
co V2T 


The changing variable to x = V2t then x? = 2t?, dx = V/2dt and the 
intagral becomes 








22h ail 2 
et V2dt=1 
ix V 20 
This implies that 
i et dt = Vr 


SO 


i e? dz = ie 


—oo 


EXERCISE (5.48). Show that the density function 


at iS ees 
f(x) — J mr/x(1-2z) 
0 otherwise, 
has distribution function with the form 
(if0<a<1,) F(x) = csin7 /z 


and find the constant c. 


5.4. SOME COMMON DENSITY FUNCTIONS 


We have 
Pi =PUX <2) 
= / f(u) du 
If x < 0, then 
Ea) = | Odu 


If0<a< 1, then 


P= i Cie [te aii 





Odu4 


-[ laa" 
=0+ f a 


ee! ‘iia 2sintcost 
0 4/sin? t(1 — sin? t) 


by the substitution sin?t =u => t =sin~! /u. In general, 

















TT 
= 2 Hee 1 Vz 
=—sin Jz. 


Compare this with F(x) = csin~! /x, we conclude that ¢ = 2/z. 


If « > 1, then 
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nafs ae Te ed mo 


0 
= Odu4 [ ut Odu 
is 0 ri 


—o4 sin VI+0 
7 
=— 1. 





Thus the required distribution function is 


if <0, 
sin -4/e OS go <1, 
ifv>1. 


Ria) = 


me aly © 


Note that F(x) increases monotonically from 0 to 1 as is required for a 
distribution function. It should also be noted that F(a) in this case is 
continuous. 


5.5. Functions of random variables 


EXERCISE (5.54). Let X be a random variable with the exponential 
distribution, parameter . Find the density function of 

(a) A=2X +5, 

(b) B= eX, 

(c) C=(1+ X)71 

(dq) D=(1+X)~°? 


SOLUTION. (a) Solution 1. If X has the exponential distribution with 
parameter 4, that is to say, 


fete) = {2 ifa <0 


Ne oP? iP ar sate 
then A = 2X +5 has distribution function 





P(A <y) =P(2X+5<y) 











p Cpa 


5.5. FUNCTIONS OF RANDOM VARIABLES 
If (y— 5)/2 < 0, then y < 5 and 


—5 


If 0 < (y—5)/2, then y > 5, and we have 


| 
—) 








y-5 
= —e 7M 2 
0 
— ee) + 1 
Thus A has distribution function 
0 ify <5, 
F = = 


Hence A has density function 


_ JO ify <5, 
= Ae 3u-) if y > 5. 


Solution 2. We have g(x) = 27 + 5, then g*(y) -_ 
5.50, 


Hagia POge®) 


d 
ere 5) 
ae 2 >) 
ese: 3(y-5) 
We have 
y-—9 
ae<0 S c= — <0 SS yK<d 
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eS Oe ee 2S a Gy 
Thus 
0 LD, 
Ha) = Vo 5) ify>5 


(b) B= g(X) = e* has distribution function 


P(B <y) =P(e* <y) 


_ JO ify <1, 
| P(X <Iny) ify>1. 


Hence fa(y) = 0 if y < 1, while for y > 1 


Thus 


0 ify <1, 
fay) = ee IF y > 1. 
(c) Let C = g(X) = (1+X)7*. Note that if X < 0, then P(X < 0) =0 
so that Pix < y,X <0) = 0. Hence we need only consider X > 0, so 
y >0C has distribution function 


5.5. FUNCTIONS OF RANDOM VARIABLES 


fay) = TPB <y) 
= FlL-MX < 2-0) 
= = a((se7G)y 
_ 5 -AG-D) 
Thus 
fey) = ee ip(=Ale- t=) aoe 


(d) C=g(X) = (14+ X)~? has distribution function 
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BB) =P aa 























d 

= —P(B 

fay) dy ( <y) 

a0 pee = | 
dy - Yy 

7 = —(1-e GY) 
A _y( he 

Thus 
fay) = Tete za i ame Fea 


EXERCISE (5.55). Show that if X has the normal distribution with 
parameters 0 and 1, then Y = X? has the x? distribution with one degree 
of freedom. 


SOLUTION. First, we seek the distribution of Y = g(X) = X?: 


5.5. FUNCTIONS OF RANDOM VARIABLES 


P(Y <y)=P(X? <y) 
ify <0 
ify >0 


0 
Sates 
= {0 ify>1 


Hence fy(y) = 0 if y < 0, while for y > 0 


d 
“ p(y 
F (Y <y) 


(if this derivative exists) 


fy(y) = 








We have 





Let t dx = 2tdt. This implies 
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ce 2 
=F Qe" dt 
0 
oat 2 
=2/ e* dt 
0 


=Vi 


since 2 {5° et dt = He e dt = V7 as f(t) =e" is an even function. 
Hence 





sees Geb 
fy (y) = ——y 2e 29 
20 
1 1 i 
— — ~% ay 
ra" “ 


0 ify <0. 


so that has the x? distribution with one degree of freedom. 


5.6. Expectations of continuous random variables 


EXERCISE (5.67). Show that a random variable with density function 


1 . 
He) = | TH if. 0 @ 1 


0 otherwise, 


has mean 5: 


SOLUTION. If X has density function 


— > Te get, 
f(z) — J m/ax(1-2) 


0 otherwise, 


then 
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aX) =f afx(e) de 


(oe) 
1 
1 
since x) = 0 for all x ¢ (0,1 = &————_ dz 
(since fre(v) ¢ (0.1)) fewer 
by the substitution sin?t = 2 => t = sin! /z for 0 < t < 7/2. Thus 
620: => t]=Cands =) SS t=772.. Thus 




















a /2 : 
a(x) = = | ee 
T JO \/sin? t(1 — sin? t) 


D) a /2 
-=| sin? t dt 
T JO 


mw/24 
= =f 1 — cos(2t) af 
0 





2 


1 1. a /2 
[Bt — 4 sin(2t)]|9 


T 
4 


NIRA; WAN > 


EXERCISE (5.68). The random variable X has density function 
(for 0 <a <1) f(a) =ce(l—2); 
Determine c, and find the mean and variance of X. 


SOLUTION. For f(x) be the density function of X, X has to satisfy 


i fx (eae = 
This becomes 


(since f(x) = 0 for all x € (—co,0) U (1, +00)) [ cx(1—2)dx=1 
0 
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This implies c = 6. 
If X has density function 


(for 0 <a <1) f(x) = 6x(1 — 2), 
then 


























Hence 
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EXERCISE (5.69). If X has the normal distribution with mean 0 and 


variance 1, find the mean value of Y = e?*. 


SOLUTION. If X has the normal distribution with mean 0 and variance 
1, then X has parameters = 0 and o? = 1 and 




















2 ee) 
= 2 =3(¢-2)" dx 
V 27 Joo 
2 ee) 
e 1,2 
= =o du 


— e€ 
V 27 Joo 





1 
since Wor f ee e~ 2” du = 1 because this is the density function of stan- 


dard normal distribution 


5.7. Geometrical probability 
5.8. Problems 


PROBLEM (5.8.1). The bilateral (or double) exponential distribution 
has density function 


(for z € R,) fo) = sco 


where c (> 0) is a parameter of the distribution. Show that the mean and 
variance of this distribution are 0 and 2c~?, respectively. 
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SOLUTION. Snce X has the bilateral (or double) exponential distribu- 
tion with parameter c (> 0), it follows that 














en A 
= i a—ce“l*l dx 
wk 


1 
=0 since the function f(x) = ace is odd, 


and 














E(X?) = ig x? f (x) dx 


—oo 


22 i} 
= fi x2 —ce ll dr 
nya oD 


CO 
a a xe © dx. 
0 


Let [ = toe xe“ dx. We integrate by part. Let u = 2? and dv = e~™ dz. 


Then du = 2a dz and v = —te®, By integration by parts, we have 
1 Sc Dips 
i= ge S| - | ce ae 
0 0 





c 
DQ [% 
a ze “dz 
c Jo 


Let Ig = ie ze“ dx. We integrate by part. Let p= x and dq = e~™ dz. 
Then dp = dx and q = —te~™, Using integration by parts, we have 





1 a: all) 4f.8 Tf 
Ip = —-te | + | e “dx = : e “dx 
Cc 0 Cc 0 Cc 0 
— 1 eel _ 1b 
(om i oa 





Thus 
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Co 
LOS) cf xe dx 
0 


Hence 








var(X) = E(X*) — E(x)? 




















PROBLEM (5.8.2). Let X be a random variable with the Poisson distri- 
bution, parameter A. Show that, for w = 1,2,3,..., 


P(X > w)=P(Y <A), 


where Y is a random variable having the gamma distribution with param- 
eters w and 1. 


SOLUTION. For w = 1,2,3,..., 


P(X >w)=1-P(X <w-1) 
hie pes 
= —X k 
k=0 


Since Y has the gamma distribution with parameters w and 1, it follows 
that Y has density function 


n= a if y > 0, 
e 0 if y <0, 


where I'(w) is the gamma function, defined by 


rw) = f gle" da. 
0 
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Thus 





Hence 


Let 
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Integrate by parts with u = x” and dv = e~* dx. Then du = nx”! dx and 
v=-—e *. Thus 


» 
=a 2]28 
I, = —ax"e B+ f a” —te—* dx 
0 


SNe Ma nh 4 


Divide both side by n!, we obtain 








al” - n! = (n — pir - 


Remember that 


1 Xr 
PY <A)= <a. ger dy 


1 


SS SS hips 
(w — 1)! ; 


Apply (**) for this equation, we obtain 











1 I Nu-1e-A 1 I 
G= pr? wait” wa" 
1 Pca aac 1 
ie ie 
GD a. aD SB 
1 ie 
gf? =~ oy TA 


Add all above equations side by side and reduce, we obtain 
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P(Y <A) = ae pee 
Neal eA NU 2 67> \e7> 
+ I 
(w—1)! (w—2)! 2! ; 
w-l yk -r 
Se a 7 Nets re +1 since 1; = sent aierh +1 
k=2 : 
aaa wal re 
k! 
k=0 
w-l1 1 
= =X k 
k=0 
= P(X > w) 


as required. 


PROBLEM (5.8.3). The random variable X has density function pro- 
portional to g(x), where g is a function satisfying 


ele ar lel SA 
w= || "if [a 


0 otherwise, 


and n (> 2) is an integer. Find and sketch the density function of X, and 
determine the values of n for which both the mean and variance of X exist. 


SOLUTION. The random variable X has density function 


oe oe if |e) > 1, 


0 otherwise, 


for some constant c. For n (> 2) is an integer, density function integrates 
to 1, so that 
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-1 oe) 
= / c(—x2)~" dx + / cx" dx 
—oo a 







































































(—a2)-"+1 =I ek [ree 
: eA Nes ' =n+1 1 
_ c c 
~  =nt1  -n+1 
2 2 
n=l’ 
giving that c= (n — 1)/2. Clearly, 
(by (5.57) E(X) = / eaves 
—1 lee) 
=i re(—2) "de + f cca le 
—oo 1 
(since fx) = 0; 
(by (5.59) E(X2) = / Pycaias 
ae lee) 
=| eC)" a+ [ aca” dx 
—oo 1 
_ é c 
~ B8=-n 38-n 
_ 2¢ 
n= 3’ 
var(X) = E(X?) — E(x)? 
ee 
MAB 


whenever the mean and variance of X exist, giving that n > 4. 


PROBLEM (5.8.4). If X has the normal distribution with mean 0 and 
variance 1, find the density function of Y = |X|, and find the mean and 
variance of Y. 


182 5. DISTRIBUTION FUNCTIONS AND DENSITY FUNCTIONS 


SOLUTION. Since X has the normal distribution with parameters = 0 
and o? = 1, that is, X has density function 


fx(z) = ee for zx ER, 


and g(x) = |a|, it follows that Y = g(X) = |X|has distribution function 


P(Y <y)=P(\X| <y) 
. ify <0, 


P(i-y<X<y) ify>0. 


Hence fy(y) = 0 if y < 0, while for y > 0 


= ax) — Fx(-y)] 


= fly fxl-9) 

= 2fx(y) since fx is even 

aleiy 

V2 
ee 


2 2 
Vo ; 


2 


Thus 


‘. if y < 0, 


[te ify >0. 


Hence 


(by (5.57) 


and 


(by (5.59)) 
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E(Y) = i yfy (y) dy 
0 T 

Af 

=\/=, 

ny) = / y fy (y) dy 


OE ag 2 1,2 
=| y\/—e 2” dy 
0 T 


= 








giving by (5.62) that the variance of Y is 








var(X) = E(X*) — E(x)? 




















PROBLEM (5.8.5). Let X be a random variable whose distribution func- 
tion Fis a continuous function. Show that the random variable Y, defined 
by Y = F(X), is uniformly distributed on the interval (0, 1). 


SOLUTION. We have that 


ImY = {F(X(w)) :w € O} = (0,1) 


since F'x(x%) + 0 as x — —oo by (5.6) and Fx (x) > 1 as x > ~w by (5.7). 
Hence P(Y < y) =Oify<Oand P(Y <y)=lify>1. ForO<y<1, 


POY y= 


(since) = 


Thus 














P(X < F71(y)) F is non-decreasing 
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0 ify<0, 
Fy(yy=<y if0<y<1, 
Labi , 


PROBLEM (*5.8.6). Let F be a distribution function, and let X be a 
random variable which is uniformly distributed on the interval (0,1). Let 
F-' be the inverse function of F’, defined by 


F7l(y) = inf{x : F(x) > y}. 


Show that the random variable Y = F~1(X) has distribution function 
F. This observation may be used in practice to generate pseudorandom 
numbers drawn from any given distribution. 


SOLUTION. If X has distribution function F’, and 


=nt{e : Fer) > y}, 
then Y = F~'(X) has distribution function 





Fy(y) =P(Y <y) 





I 
al 
by 
S 
IA 
= 











< F(y)) since F' is non-decreasing 


since 0 < F(y) < 1 for all y and X is distributed on the interal (0, 1). 


PROBLEM (5.8.7). If X is a continuous random variable taking non- 
negative values only, show that 


E(X) = i; Rea as 


whenever this integral exists. 














SOLUTION. We shall prove that 














E(x) = is P(X > x) dz. 
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We have that 


i P(X > 2) dx 





cog by integration by parts formula with 
= ¢P(X > aie - f v7 (P(x > x)| dx u=P(X > «2),dv =dz 
9 ia du= 4[P(X > 2x)|dz,v=2 


= Jim a[l — Fx(x)] — p x{l — Fx(x)]' dx 


= Jim a[l — Fx(ax)] + I xufx (x) da. 











Since 
lim x[1— Fx (x)] = lim - 
— X: = 
t400 none FG) 
TW An} ’ = ih 2 
(by l’Hépital’s Rule) Jim fx (2) 
[1-Fx (@)P? 
> 2 
ages [1 — Fx(z)] 
LOO fx (x) 
= 0, 


it follows that 














Hence 














as required. 
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PROBLEM (*5.8.8). Use the result of Problem 5.8.7 to show that 


2(9(X)) = f "a epetaas 


—oo 














whenever X and g(X) are continuous random variables and g: R — [0, 00). 


SOLUTION. Applying the conclusion of Problem 5.8.7 to Y = g(X), we 
obtain 





























as required. 


PROBLEM (5.8.9). The random variable X’ is said to be obtained from 
the random variable X by truncation at the point a if 


, _ JX(w) if X(w) <a, 
es {; if X(w) >a. 


Express the distribution function of X’ in terms of the distribution function 


of X. 
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SOLUTION. Since X has distribution function Fy, and 


a ifx>a, 


x ifa<a, 
we)= sag 


it follows that X’ = g'(X) has distribution function 


Fxi(y) =P(X' < y) 














P(a 2) if X (w) s a. 


If y <a, then Fy/(y) = 
then X'(w) =y = X(w)). 

If y > a, then Fxyr(y) = P(X’ < y) = P(X’ < a) +P(a < X’ < 
y) = Fx(a) = 1 (since P(a < X’ < y) = 0 by X’(w) < a for all w and 
(X’ <a) =Q). Thus 


P(X’ < y) = P(X < y) = Fx(y) (since y < a, 


PROBLEM (5.8.10). Let X have the exponential distribution with pa- 
rameter 1. Find the density function of Y = (X — 2)/(X +1). 


SOLUTION. Since X has the exponential distribution with parameter 
X, that is, X has density function 


e* ifa>0, 
Ix(#) = ‘6 if x <0, 


and g(x) = (x — 2)/(a + 1), it follows that Y = (X — 2)/(X + 1) has 
distribution function 
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P(-1< X < 4”) if -2<y<1, 





: ify <—-2ory>1, 


y 
Hence fy(y) = 0 if y < —2 or y > 1, while for -2<y<1 








Thus 





Bo 4 
poate? Bates 
0 otherwise. 


PROBLEM (*5.8.13). A unit stick is broken at n random places, each 
uniform on [0,1], and different breaks are chosen independently. Show that 
the resulting n + 1 substicks can form a closed polygon with probability 
1—(n+1)/2n. 


SOLUTION. Let X; be the position of the Ath break (in no special 
order). The pieces form a polygon if no piece is longer than the sum of 
the other lengths (since the straight segment connects two distinct points 
is the shortest among the curves), which is equivalent to each piece having 
length less than $ (since if we denote the length of ith piece by x;, then 
Uy < y+ ag +--+ +4j-14+2j41+-+-+ 4p, and so 2x; < yaa x; = 1, giving 
vi < 5 for 7 = 1,2,...,n). This fails to occur if and only if the disjoint 
union Ag U A; U---U A, occurs, where Ag is the event there is no break in 
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(0, 3], and Ax is the event of no break in (X,,X,+ 4] for k > 1 (remember 
the permanent break at 1). Now, 





1 1 1 
P(Ao) = P(X > -,X9> =,...,Xn > 5) 











2 2 
1 1 1 
= P(X > 5) P(X2 > 5) ee PEUX 3) by independence 
1 n 
_ (5) ’ 
and for k > 1, 


P(Ax) 


0 
= ee P(X; € [0,\(Xn, Xe + 5) for ¢ £ k) dx 
0 


- is [[PC% € (0, 1\(Xe, Xe + ») dx 


0 itk 
2 1 n-l1 <, 1 1 , 
= / (=)"" dx since P(X; € [0,1]\(Xxn, Xn + =)) = = fori Fk 
ao) 0 8 
lin 
=(5)". 
Thus 
P(LJ A;) = S > P(Aa) by disjointness 
i=0 i=0 
1 
Gay 


Qn 
Hence the probability that the resulting n + 1 substicks can form a closed 
polygon is 


PU Aj)°) =1— P(U Aj) 
sa Se 
an 


as required. 
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PROBLEM (5.8.14). The random variable X is uniformly distributed on 
the interval [0,1]. Find the distribution and probability density function of 
Y, where 


(Cambridge 2003) 


SOLUTION. Since X is uniformly distributed on the interval [0,1] with 
distribution function 


0 ifx<0, 
Fy(e)=<2@ 20K 2 <1, 
1 ife>l, 


and g(x) = 3x/(1 — 2), it follows that Y = 3X/(1— X) has distribution 
function 





Fy(y) =P(Y <y) 











40 if y <0, 
Pye) diye 0. 


-\° ify <0, 


73 ify > 0. 


Hence fy(y) = 0 if y < 0, while for y > 0 
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fry) = “FV 
- d 
dy 


—-P(Y <y) 


Thus 
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Part 2 


Further Probability 


CHAPTER 6 


Multivariate distributions and independence 


6.1. Random vectors and independence 


EXERCISE (6.13). Show that two random variables X and Y are inde- 
pendent if and only if 


PX >a, Y¥ >y). =P XX S>a)PY > y) for z,y ER. 


SOLUTION. Suppose that A and B are two independent events. Then 


P(AN B) = P(A)P(B) 





























So 

P(A B°) = P((AUB)’) by De Morgan’s law 
=1-P(AUB) since P(AN B) = P(A)P(B) 
= 1-[P(A) + P(B) —-P(AN B)] 
= 1— P(A) — P(B) + P(A)P(B) 
= (1— P(A))(1 — P(B)) = P(A®)P(B*) 

Therefore, 


P(A°N B°) = P(A°)P(B°). 


Thus A° and B® are two independent events. It follows that A and B 
are independent if and only if A° and B® are independent event (since 
(4°) = A, (B®) = B) 

For A = (X > 2), B=(Y > y), X and Y are independent random 
variables iff (X <x) and (Y < y) are independent random variable, thus 
X and Y are independnent iff (X > c) and (Y > c) are independent events, 
that is, PX Sey Sy) = PCS > 2yPY Sy) 
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EXERCISE (6.14). Let the pair (X,Y) of random variables have joint 
distribution function F(x, y). Prove that 


P(a< X <b,c< Y<d) = F(b,d) + F(a,c) — F(a,d) — F(b,c) 
for any a,b,c,d € R such that a < band c < d. 


EXERCISE (6.15). Prove that two random variables X and Y are inde- 
pendent if and only if 


Pia<X<b,c<Y<d)=P(a<X <b)P(c<Y<d) 
for all a,b,c,d € R satisfying a < b and c < d. 


SOLUTION. Suppose that X and Y are random variables with joint 
distribution function F(x, y). Then 


F(b,d) = P(X <b,Y <d)=P(X < b)P(Y <d) 
From Exercise 6.14, we have 


Pia< X <b,c< Y<d) = F(b,d) + F(a,c) — F(a,d) — F(b,c) 
























































= P(X < b)P(Y < d)+ P(X < a)P(Y < cc) —P(X < a)P(Y < d) 
—P(X < b)P(Y <c) 

= P(X <a)[P(Y <c)-P(Y < d)|- P(X < b)IP(Y <c) -P(Y < d)] 

= [P(X <a) P(X < d)IIP(Y <0) -P(Y < a) 

= [P(X <b) -P(X <a)|[P(Y <a) -PYY <0) 

=P(a< X <b)P(e< Y <d) 





as required. 


6.2. Joint density functions 


EXERCISE (6.25). Random variables X and Y have joint density func- 
tion 


c(a? + 52ry) fO<ar<10<y<2, 
f(y) = 
0 otherwise. 


Find the value of the constant c and the joint distribution function of X 
and Y . 


SOLUTION. Joint density functions integrate to 1, so that 


6.2. JOINT DENSITY FUNCTIONS 


II 
° 
Ss 
2 
— 
No 
8 
bo 
+ 
8 
Sa 
Q. 
8 


II 
io) 
—~ 


giving that c = 6/7. The joint distribution function of X and Y is 


x y 
P(X <2,y<y)= a flu,v) dude 
u=0 Jv=0 


Ay (u (uw? + Suv) du de 
u=0 Yv= 


= gt u+ qu o=o du 


= 38 ‘s 2 1 2 
=F f- jY y+ quy") du 


wu 1 
Wig gh tde: 0 


6 
7 
Gay 22 
“(= + ey’). 


Thus 


S(da%y + day?) if0<ar<10<y< 2, 
F(z,y) = 
0 otherwise. 
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EXERCISE (6.26). Random variables X and Y have joint density func- 
tion 


e* 4 ifa,y>O0, 


0 otherwise. 


fay) = 
Find P(X + Y <1) and P(X > Y). 


SOLUTION. Suppose that X and Y have joint density function 


e*Y ifa,y>0, 


0 otherwise. 


Then 


P(x +¥ <i= ff fleydedy 


by Theorem 6.22, where A = {(z,y) € R?: «+ y < 1}. Writing in the 
limits of integration, we find that 


1 1—x 
PX+Y<1)= / / e Yardy 
z=0 Jy=0 


Moreover, 


px+y<si= ff fleydedy 


by Theorem 6.22, where B = {(a,y) € R? : x > y}. Writing in the limits 
of integration, we find that 
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PX <Y)= / / er da dy 


z=0 Jy=0 


II 

| 

| 
+ 
a 


6.3. Marginal density functions and independence 


EXERCISE (6.35). Let X and Y have joint density function 


cn) ce fO<y<a<l, 
x, = ° 
# 0 otherwise. 


Find the value of the constant c and the marginal density functions of X 
and Y. Are X and Y independent? 


SOLUTION. Joint density functions integrate to 1, so that 


i= / f(x,y) dx dy 
fie ae 
=t | cx dx dy 
«z=0 Jy=0 


1 
= ef ry|j—o az 
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giving that c= 3. The marginal density function of X is 


Or / ~ Pua 


-\eew if0<a<1, 


0 otherwise, 


= 3zylj-o fO0<2<1, 
0 otherwise, 


=: SG") alO-e eet, 
~~ \0 otherwise, 


and the marginal density function of Y is 


fru)= f f@u)ae 
7 J, 3x dx iO gre, 
~~ 10 otherwise, 


if0<y<l, 


otherwise, 


0 
_ fs-y?) f0<y<1, 
0 otherwise, 


Clearly, X and Y are dependent, since 


3x?(1—y?) if0<y<a<1l, 


0 otherwise, 


fx(@) fy (y) = 


. if0<y<a<l, 


0 otherwise, 


6.3. MARGINAL DENSITY FUNCTIONS AND INDEPENDENCE 201 


EXERCISE (6.36). Random variables X, Y, and Z have joint density 
function 


8ryz if0<a2,y,2 <1, 
f(x,y, 2) = 
0 otherwise. 


Are X, Y, and Z independent? Find P(X > Y) and P(Y > Z). 


SOLUTION. The marginal density function of X is 


=f f teneavas 


ve ee pStyzdydz if0<a2<1, 


otherwise, 


hp 4892? | 25 dy ifO0<2<1, 


otherwise, 


i cee 
= 


otherwise, 


xy Zl, if0<a<1, 
0 otherwise, 
7 if0<a<1, 


0 otherwise. 


The random variable Y and Z have this marginal density function also, and 


f(2,y, 2) = fx(x) fy (y)fz(z) for r.yER, 


so that X, Y and Z are independent. We have that 
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p(x >¥)= fff f(a, y, z) dx dy dz 
ABDA'B'O 
1 he 1 
= ) / i 8xyz dx dy dz 
«=0 J y=0 ¥ z=0 
1 c 
= ‘| i Axyz*|*_» dx dy 
«=0 J y=0 
1 x 
= I / Axy dx dy 
«=0 J y=0 
1 
=a 2cy" [7-0 dx 
x=0 


1 

= 2x° dx 
0 
4 


aaa 
= S10 


Nile 


Similarly, P(Y > Z) = 1/2. 


PY 7) = rast ise Ores z) dx dy dz 


1 1 y 
= / | if 8xyz dx dy dz 
«z=0 J y=0 ¥ z=0 


6.4. Sums of continuous random variables 


EXERCISE (6.45). If X and Y have joint density function 


1 —x“— ‘ 

s(~t+y)e*” 4 ifa,y>0, 
f(x,y) = 12 +9) 

0 otherwise, 


find the density function of X + Y. 


SOLUTION. The sum Z = X + Y of two jointly continuous random 
variables has density function 


6.4. SUMS OF CONTINUOUS RANDOM VARIABLES 203 


(by (6.37)) fale) = / ee ce ne 
fo f(@z—-u)du ifz>0, 
0 


otherwise, 


since f(u,z—u)du =0 unless u > 0 and z—u> 0. Thus, for z > 0, 


fae)= f 520 du 


1 —z z 
= 52e ulg 
1 22% 

= 9° e€ 


Hence 
1,2,-z B 
pee e if z > 0, 
z —_ 
fal) f otherwise. 


EXERCISE (6.46). If X and Y are independent random variables having 
the y? distribution with m and n degrees of freedom, respectively, prove 
that X + Y has the y? distribution with m +n degrees of freedom. 


SOLUTION. Z = X + Y has density function 


(by (6.39) fale) =f fx@)fr(e- 2) az 
: fx(a)fy(z-—a)dx ifz>0, 
0 


if z <0, 


since fx(x) fy(z — x) =0 unless ¢ > 0 and z—a2 > 0. Thus, for z > 0, 


= ° u t im—-1,—t4a2 1 L = tn—1,—4(z-2) 
fae)= f (km) 27)” e 2 mn 2? | liar dx 


z 
1 1 1 
=a f gamle 2% (zg — 27 le2(2-2) gy 
0 


NIF 


where ??? 
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EXERCISE (6.47). If X and Y are independent random variables, each 


having the normal distribution with mean 0 and variance 1, find the distri- 
bution of X + Y. 


SOLUTION. Let X and Y be independent random variables, each having 
the normal distribution with mean 0 and variance 1. Then X and Y have 
dendity functions 








fic(0) = T= exn(—52"), 
fyy) = Fae ox(—50") 














f{2ziz) = fx(x)fy(z — 2) dx 
fe A Age “a 1 : 
fee 5 OPI 5t ips 5 (2 x)") dx 
Bey er ee ae ee 
a a p( 5t 5 (2 0)" ae 
pe ae i 2 19, 1 2 
=e Pa gE — 5% t2e— 5 ) dx 
2 tae 2 1 5 
a ae exp(—2* + zx — 37 ) dx 
i fe san es ee 
=e oe: p(-—(« 5%) rhe ) dx 
= i, exp(—(a — =z)”) dx 
where 
1 ae 
A= an CxP(-7? ie 


Substitute u = a — 52 in the last integral to obtain 
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fz2)= af” exp(—u?) du 
1 


1 
(since [°° exp(—u*) du = V7) =5 mexp(—72") 
T 
1 1, Zz \9 
= ex 

5 
The only distribution with density function of the form above is the normal 
distribution with with mean 0 and variance 2. 





6.5. Changes of variables 


EXERCISE (6.54). Let X and Y be independent random variables, each 
having the normal distribution with mean p and variance a7. Find the joint 
density function of U = X—Y and V = X+Y. Are U and V independent? 


SOLUTION. The density function of X is 


fx a) = poms oxl-5g 


and the density function of Y is 


(x — p)*), 








fr (2) = seas ex sry 4). 


The random variables X and Y are independent, so that the joint density 
function of X and Y is 


fxy (x,y) = fx(2) fr (y) 











1 1 en . 

— exp x exp y 
Fae (53 ty) ae rsa - #2) 

ee 1 2_ 1 2 

= 5752 OxP(-grale — HY" — Sy — #)”). 


The mapping T of this problem is given by T(z, y) = (u,v), where 


U==-Y, v=ur+y, 


and T is a bijection from D = {(z,y):2,y € R} to S = {(u,v): u,v € R} 
(or from D = R? to S = R*). It has inverse T~!(u,v) = (x,y), where 


206 6. MULTIVARIATE DISTRIBUTIONS AND INDEPENDENCE 


The Jacobian of T~! is 


Ox Ox 
ni 
Ou Ov 


giving by Jacobian’s formula that U and V have joint density function 


fu,v(u, v) 
1 1 


fx,y(z(u, v), y(u, v)) | J(u, v)| 


1 
iS g(-u tv). 


NS) Ts SO) Lael 





~ 2° Wo? 


yu)? 


o2 








2 


UU 


*(y—1)) 





+ py? — up — vp 


up + 27) 








= Fag OPC G53 (Gq) DOP 


2 


U 


1 


202 
1 1 


v—2p9 
a) 





1 
- exp( 


2m (207) 202 °2 


u*)) 


ex 
21 (207) 


so that U and V are independent. The functions 


i 





and 


(Su) 


1 ju- 2H y2y 





gv(v) = 


27 (207) 


exp( 


Io! J2 


are density functions that respectively, having the normal distributions with 
mean 0 and variance 207, and mean 2p and 207. 
The marginal density function of U is 
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fu(u) = a fu,v (u,v) dv 
= gu(u) i: gv (v) du 


(since [°° gv(v) dv = 1.) = gu(u). 


Similarly, the marginal density function of V is 


fv(v) = i fu,v (u,v) du 


(since f°. gu(u) du = 1.) = gy (v). 
EXERCISE (6.55). Let X and Y be random variables with joint density 


function 
flw,y) = ta ayy > 0, 


= gv(v) p: gu (u) du 


0 otherwise. 
Show that the joint density function of U = $(X —Y) and V =Y is 
vee se"? if (u,v) € A, 
paeanl F otherwise, 


where A is a region of the (u,v) plane to be determined. Deduce that U 
has the bilateral exponential distribution with density function 


fu(u) = se 


SOLUTION. The mapping T of this problem is given by T(x, y) = (u,v), 
where 


1 
u=5(t— 4); Vv=Y, 


and T is a bijection from D = {(2,y): z,y > 0} to S = {(u,v) : u > 
—5v,0 <u < oo, }. It has inverse T~1(u, v) = (2, y), where 


r=2ut+, yY=v. 
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The Jacobian of T~! is 


fe) 0. 
JE E|=[o i fe? 
a ae 0 1 


giving by (6.51) that U and V have joint density function 


fxy(z(u, v), y(u, v)) |J(u,v)| ifu > —fv and 0<vu<o, 


0 otherwise, 


fu,v(u, v) = 


= aa |2| ifu>—jv and 0<u<o, 
~ (0 


otherwise, 


_ se ute) ifu>—jvand0<v<, 
0 otherwise. 


For u > 0, 


fu(u) = i fu,v (u,v) dv 


0 CO 
=f fuytur)dv+ [ fov(ur) de 
ae 0 


For u < 0, 
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fulw) = f fu,v (u,v) du 
—2u 


Co 


= / fuv(uv)dv+ | fov (u,v) do 


—2u 


= Fee 
= 
ots 

= se 


The marginal density function of U is 


(for u € R,) fu(u) = se 


so that U has the bilateral exponential distribution. 


6.6. Conditional density functions 


EXERCISE (6.60). Suppose that X and Y have joint density function 


Aes e¥ 0<ar<y<om, 
x, = . 
y 0 otherwise. 


Find the conditional density functions of X given that Y = y, and of Y 
given that X = x. 


SOLUTION. The marginal density function of X is 
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f(a) =f fleu)ay 
= [ e 4 dy 


eo) 


for x > 0, and the marginal density function of Y is 


fle) =f fa,y) a 
y 


= [ e dz 
0 


=e *[x\5 
= e Yy 
for y > 0, where it is understood that these functions take the value 0 off 


the specified domains. The conditional density function of Y given X = x 
(> 0) is 


fxy(2,y) 
fx (2) 


e Y 


fyix(y| 2) = 


ez 
= ery 


for y > x. The conditional density function of X given Y = y is 


fxyy(@|y) = foe 


ey 





e Yy 
1 


y 


for 0 < x < y. It is clear that both these conditional density functions 
equal 0 if a > y. 
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EXERCISE (6.61). Let X and Y be independent random variables, each 
having the exponential distribution with parameter . Find the joint den- 
sity function of X and X + Y, and deduce that the conditional density 
function of X, given that X + Y = a, is uniform on the interval (0, a) for 
each a > 0. In other words, the knowledge that X + Y = a provides no 
useful clue about the position of X in the interval (0, a). 


SOLUTION. The density function of X is 


het 4b ge 0; 
pete) = {9 fo < 0, 


and the density function of Y is 


Aer” if y > 0, 
fy(y) = f if 250: 


The random variables X and Y are independent, so that the joint density 
function of X and Y is 


fxy(2,y) = fx() fy (y) 
a if >Oandy>0, 


0 otherwise. 


Let U = X and V =X +Y. The mapping T of this problem is given by 
T(z,y) = (u,v), where 


U=2, V=X+Y, 


and T is a bijection from D = {(z,y):2,y > 0} toS = {(u,v):0<u< 
co, uv > u}. It has inverse T~!(u,v) = (a, y), where 


c= U, y=-ut. 


The Jacobian of T~! is 





2) O 
Sy oy a 


or 


giving by Jacobian formula, (6.51) that U and V have joint density function 
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fuv(u,v) = ae |J(u,v)| ifu>Oandu>u, 


0 otherwise, 


Z . ifu>Oandv>u, 


0 otherwise, 


= or ifu>Oandv>u, 
— \0 


The marginal density function of V is 


for v > 0. The conditional density function of U given V = a (> 0) is 


= fuv(u, a) 


fuiv (u | a) fv (a) 


ere 
~ \2eAaq, 


for 0 <u<a. Thus the conditional density function of X given X+Y =a 
(> 0) is 
1 
Fxixty (a, a) = © 


for 0 < a <a, so that X given that X + Y =a is uniform on the interval 
(0,a) for each a > 0. 
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6.7. Expectations of continuous random variables 


EXERCISE (6.72). Let the pair (X,Y) be uniformly distributed on the 
unit disc, so that 


a! ifar@+y? <1, 


0 otherwise. 


fxy (a, y) = 








Find E(VX2+4+ Y2) and E(X?+Y7%). 




















SOLUTION. We have that 














E(V X24 Y2) = ie Var? +y*fxy (a, y) dx dy 


by Theorem 6.62, where D = {(a,y) : 2? + y? < 1}. Changing variables to 
polar coordinates, we find that 














a |O|gr 9 


Similarly, 
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RY 2 2: = 2 2 d d 
(x? + ¥2) [Le ca veins 


QT 1 
(by) = : r?a_'r dr dO 
6=0 J r=0 
= qt a ler dé 
e-0 4 7 
1 - 20 
=-7 dé 
4 6=0 
oe wT 
= aly "[Olax0 
_ 1 
=o 


EXERCISE (6.71). Give an example of a pair of dependent and jointly 
continuous random variables X, Y for which E(XY) = E(X)E(Y). 
































SOLUTION. Let X and Z be independent, X having the normal distri- 
bution with mean 0, and variance 1, and Z taking the values +1 each with 
probability 7 The density function of X is 








1 1 
x”) for x ER, 


ex 
5 p( 5 


and the mass function of Z is given by 


fx(z) = 











1 1 
Saar exp(—3t?) if 2 =—1, 
2 1 RD tes 3 
aan CXP( 5t°) ifz=1, 
0 otherwise. 


We have that 


6.7. EXPECTATIONS OF CONTINUOUS RANDOM VARIABLES 215 


(by Theorem 2.42) 
(XY) = E(XY | Z=-1)P(Z =-1) + E(XY | Z=1)P(Z =1) 


1 
B(X7) + 5 
































E(—X?) 




















Onwle 





























On the other hand, E(X) = 0, giving that E(XY) = E(X)E(Y). 
We have that 




















P(X <0,Y <0) =P(X <0,—X <0) 











= P(X =0) 
=0 
On the other hand, 
1 
P(X <0)=-— 
2 
and 
PY = 0) = i 
“D 


giving that 


P(X <0,Y <0) 4AP(X < 0)P(Y <0) 
which is to says that X and Y are dependent. (similar to Example 3.22) 
EXERCISE (6.72). If X and Y have joint density function 


e¥’ if0<a<y<owm, 


0 otherwise, 


Ke=4 














find E(X | Y =y) and E(Y | X =2). 














SOLUTION. The marginal density function of X is 
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f(a) =f fleu)ay 
= [ e 4 dy 


eo) 


for x > 0, and the marginal density function of Y is 


fle) =f fa,y) a 
y 


= [ e dz 
0 


=e *[x\5 
= e Yy 
for y > 0, where it is understood that these functions take the value 0 off 


the specified domains. The conditional density function of Y given X = x 
(> 0) is 


fxy(2,y) 
fx (2) 


e Y 


fyix(y| 2) = 


ez 
= ery 


for y > x. The conditional density function of X given Y = y is 


fxyy(@|y) = foe 


ey 





e Yy 
1 


y 


for 0 < x < y. It is clear that both these conditional density functions 
equal 0 if a > y. 
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We have that 


BX|¥ =a) = f- nfo (e | y) de 


217 


and 


BY |X=2)= [ 


(oe) 


yfy|x(y | x) dy 


ie, @) 
i ye" 4 dy 
zx 


CO 
= e | ye "dy 
x 


= e{-ve "pe, +f e¥dy} 


x 


=e" "4e *) 
=e"e “(x4+1) 
=at+l. 


6.8. Bivariate normal distribution 


EXERCISE (6.79). Let the pair (X,Y) have the bivariate normal density 
function of (6.76), and let U and V be given by (6.78). Show that U and V 


have the standard bivariate normal distribution. Hence or otherwise show 


tha 


and that 














E(XY) —E(X)E(Y) = poise 


























E(Y | X) = pa + poo — p1)/o1. 


SOLUTION. The pair (X,Y) has the bivariate normal density function 
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(for x,y € R,) fxy(z,y) = 
270102\/1— p 


where Q is the quadratic form 





are ace ig ore eras 


and 1, Mo € R, 01,02 > 0, -1<p<1. Let 








The mapping T of this problem is given by T(z, y) = (u,v), where 


_ &— fI ee Pore 
O1 , 02 , 


and T is a bijection from D = {(z,y): x,y € R} to S = {(u,v) : u,v € R}. 
It has inverse T~'(u,v) = (x,y), where 


L=out /[, Y = 02U + pa. 
The Jacobian of T~! is 








Ox Ox 
Qu gv | —| 71 | ai 
Yy Yy = = 0102, 
ear 0 o2 
giving by Jacobian formula, (6.51) that U and V have joint density function 


fu (u,v) = fxy (eu, v), yu, 0) |J(u, ¥)| 





= 1 Z Te (u?—2puv+v?) 


270102\/1— p? 





- 1 : saat 2puv+v?) 


27/1 — p? 
so that U and V have the standard bivariate normal distribution. 
We have that 
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E(XY) — E(X)E(Y) = E((oiU + p2)(92V + p2)) — E(oiU + p2)E(o2V + pe) 
= E(o102U + o1piaU + o2jnV + pipli2) — [oi E(U) + p1)\[o2E(V) + p2] 
= P0102 + 12 — Pipe 

















= po102, 


as required, since E(UV) = p and E(U) = E(V) =0. 


The conditional expectation of Y given X = z is 












































E(Y | X =2) =E(ooV + pg | mU + 1 = 2) 




















1 bcos 
= E(o2V + po |U = am at) 
O71 














Gia 
Gh |S ais 
O71 


tv HI 





= 020 + pa, 


























as required, since E(u2 | U = x) = we and E(o2V | U =2) =o2E(V |U = 
XL) = onpx. 


EXERCISE (6.80). Let the pair (X,Y) have the bivariate normal dis- 
tribution of (6.76), and let a,b € R. Show that aX + bY has a univariate 
normal distribution, possibly with zero variance. 


SOLUTION (1). The pair (X,Y) has the bivariate normal density func- 
tion (6.76), that is, 


1 
(for x,y € R,) fxy(z,y) = e732 O(ay) 
270102\/1— p 





where Q is the quadratic form 





1 (®@ 2 Hye ff) Y~H2, | (YT Haye 
Y= 2 
Q(0.9) = yl - HE) +E 
and 41, M2 € R, 01,02 > 0, -l<p<l. 
Firstly, we shall show that if T has the normal distribution with mean 
pi and variance o”, then kT, where k is a real constant, has the normal 
distribution. If k > 0, then the distributuion of kT is 
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Fryr(2) = PRE < a) 


=P(T< 7) 


1 ae 1 5 
= += | exp(—s a(t — p)”)) dt. 


In this case, the density function of kT is 











fier(2) = Fir(2) 


(cz 
202k 





= + exp( ?) 
kV 210? P e 
1 





1 
Tako? exp( Ako ku)”) 


so that kT has the normal distribution with mean ky and variance (ko)?. 
If & < 0, then the distributuion of kT is 





Fyr(a) = weg < x) 


=P = 7) 


1-P(T <5) 











1 nee 1 3 
== ca exp(—5 a(t — H) )) dt. 


In this case, the density function of kT is 


fer (x) = © Fer(2) 


1 
= exp 
kV 21a? ( 20 
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so that kT has the normal distribution with mean ky and variance (—ko)?. 
If k = 0, then kT has the normal distribution, so long as one allows de- 
generate normal distributions with zero variances. The converse is true, so 
long as one allows k £0: T = (1/k)(kT). 

Secondly, we shall show that if T has the normal distribution with mean 
pi and variance a7, then T + a, where a is a real constant, has the normal 
distribution. In fact, the distribution function of T+ a is 














Friq{(xz) =P(T +a < 2) 
=P(T<z-a 
a 


r— 


ss |. oval mat 


The density function of T + a is 
d 
fer(2) = 7 Fr+a(2) 
ee | 
= AF exp 


210 


1 


sa (w- 0-1") 








1 1 4 
—_ /2no2 exp( 202 [x (a + LW) ) 
so that T’+ a has the normal distribution with mean a+ js and variance 
(ko)*. The converse is true: T= (T +a) + (—a). 
Let U and V be given by (6.78), that is, 


poo, V= 

O71 02 
Then aX + bY = aoyU + bogV + (apy + byu2). We can assume that a 4 0 
and b £ 0. Using the above two facts, we deduce that aX + bY has the 
normal distribution if and only if aoyU + bo2V has the normal distribution. 
But aa,U + bo2V has the normal distribution if and only if U +aV, where 
a #0, has the normal distribution. The distribution function of U + avV is 





Fusav (z) = P(U t+aV < Zz) 
(z-u)/lal poo 
= / i Fu y(u, v) du dv, 


v=—0o =—0o 
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where 


1 1 
Fuy (u,v) = exp u? — 2puv + v?)). 
ey Qn/1 — p2 ( a — py y ) 








The density function of U + aV is 


fu+ev (2) = sO eau) 














dz 
aha i : 1 2 Z—U, | /Z—Ue 
eres sa py 2pu( a Jat A )"]) du 
= al oo 1 2p, 1,2 2pz , 2z 2 
= ace faa ltt Bt ae - CE + Bot Se 
= af exp(— 5 (av? + bu +u)) du 
where 
ag, Oey 
2m jal V/1— p?’ a?(1— p?) ’ a2(1 — p?)’ a2(1 — p2)’ 
We have that 
fusav (2) = sf exp(—= (au? + bu + c)) du 
w= as ) 
b2 
= 5 vax + dex 
maa sar en Tal) 


Substitute t = /au + Ia in the last integral to obtain 
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oe pe ere 
met Qn |al /1 — p? Jno Va ae 4a, 
1 1 b? i ait 
= xX c e 2 dt 
nae ea p(—5(e- Go) o | 





1 Vin 1 Ze 4(1 + pa)?2z? a? (1 — p”) 


























_ cag Vi — pe (aes exp( alae” — p?) at(1 — p/P : A(a2 Pops i)! 
a? (1—p?) 
1 1 o2(1 — p2)z? 
— exp( [ 5 ( a ) ] 
Qn (a2 + 2pa +1) 2°a?(1 — p?)(a? + 2pa 4+ 1) 
1 1 . 
= exp( 5 2) 
Qn (a? + 2pa +1) 2(a? + 2pa +1) 





so that U + aV has the normal distribution with mean O and variance 
(a? + 2pa 4+ 1). 


SOLUTION (2). By changing of variables as in (6.78), the pair (X,Y) 
has the standard bivariate normal distribution. We shall first find the joint 
density function of U = X and V = aX + bY by the method of change of 
variables, and then find the marginal density function of V. 

We consider first the case b 4 0. The mapping T of this case is given 
by T(a,y) = (u,v), where 


U=Z, v=ax + by, 
and T is a bijection from D = {(z,y): x,y € R} to S = {(u,v) : u,v € R}. 
It has inverse T~!(u,v) = (x,y), where 
a 
r=u, y= rut —v. 


The Jacobian of T~! is 





dn 8 

| a Di eats 
Ds BON We cB. lin Be 
Ou Ov bob 


giving by Jacobian formula, (6.51) that U and V have joint density function 
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fu (u,v) = fxy (eu, ¥), yu, »)) [J (u, e)| 











1 2 2 1 
st 1 on Wa apey (1+ 208 + Fou? oe (pba) ev?) 


For m > 0, then 


aa 1 2 ee weed ae Cay me int 
/ enBlmw+fut9) gy = : eoalWiit se t+9-£ a, 





—oo —Co 
2 oe) 
£30 fm) f 2h dt 
oo Vm 
ven .-3(0-£) 
= ——eE 2 4m 
vm 
The marginal density function fo V is 
[o-e) 
ful) =f fov(ue)du 
—Co 
1 v2 4(pb+a)? 2 
1 V 20 Hap lb al(14292+9) 








= . e 
2 |b| Vi — p2  va?-+2pab+b? 
|b|\/1—p? 











2,2 2 
pi b*+2pabta 2 
= 1 e 2(1—p2)b2 (1 b24+2pab+a2 yu 
V 20 \/a? + 2pab + b? 
1 (1=p?)b? 9 





3 Vv 
e 2(=p2)b? 62+2pab+a? 











7 V2r1/ a? + 2pab + b2 
1 = 1 y2 

= eo He Aaah TP 

/2r(a2 + 2pab + b?) 


so that V = aX + bY, where b 4 0, has the normal distribution with mean 
0 and variance a? + 2pab + b?. 

We consider next the case 6 = 0. In this case, we could do the same as 
in the first case; however we can use the moment generating function the 
shorten the problem. The moment generating function of V is 
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My (t) = E(e***) 

















where 

Mx(t) =e72", 
giving that 

My (t) = e2? ©, 


which we recognize by (7.58) as the moment generating function of the 
normal distribution with mean 0 and variance a?. 


6.9. Problems 


PROBLEM (6.9.1). If X and Y are independent random variables with 
density functions fx and fy, respectively, show that U = XY and V = 
X/Y have density functions 


ful) = f * fx(a)fr(u/a)r ax, fulv)= f > ae) utdi: 


SOLUTION. If X and Y are independent, then fx y(x,y) = fx(x) fy (y). 
For the first part, we shall find the joint density function of W = X and 
U = XY, and then find the marginal density function of U. The mapping 
T of this part is given by T(x, y) = (w,wu), where 


w=, uUu= cry, 


and T is a bijection from D = {(z,y): 2 €R,y € R} toS={(w,u): wF 
0,u € R}. It has inverse T~!(w, u) = (x,y), where 





U 
L=U, y=. 
w 
The Jacobian of T~! is 
0. (6) 
om Bu | -| r. 2 | eee 
oy Yy uw 1 ’ 
dw bu we w W 
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giving by Jacobian formula, (6.51) that W and U have joint density function 


fwu(w,u) = oo |J(w,u)| if w 40, 


0 otherwise, 


0 otherwise. 


S iaial aes ose 


The marginal density function of U is 


fu(u) = i fw (w, u) dw 


1 
w 


= [© txcuytvtuju)| 4 du 
= [ fxlo)fv (ula) ae 


as required. For the second part, we shall find the joint density function of 
W =Y and V = X/Y, and then find the marginal density function of V. 
The mapping T of this part is given by T(z, y) = (w,v), where 


w= Y, Pea 


and S is a bijection from D = {(z,y):2€ R,y€ R} toS={(w,v):wF 
0,v € R}. It has inverse T~!(w,v) = (x,y), where 


r= wy, y= Ww. 


The Jacobian of S~! is 





Ox = Ox 
¥y ¥y ’ 
Ow Ov 1 0 


giving by Jacobian formula, (6.51) that W and V have joint density function 
es eae 


0 otherwise, 


0 otherwise. 


Bf eure if w £0, 
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The marginal density function of V is 


fv(v) = [- fw (w,v) dw 
= [- fx (wv) fy (v) |-w| dw 
7 a i fx (vy) fy Y) yl dy da 


as required. 


PROBLEM (6.9.2). Is the function G,, defined by 


1 ifet+ty>0, 
ate.) =| : 


0 otherwise, 


the joint distribution function of some pair of random variables? Justify 
your answer. 


SOLUTION. No. Clearly, 


2 
Ox Oy 


for all x and y, and it is the case that 


co [ee) 62 oo Co 
If. Fea cendedy= ff odeay 
=0 


# i, 


G(x, y) =0 


giving that pg G (a, y) is not a joint density function, so that G(z, y) is 
not a joint distribution function. 
PROBLEM (6.9.3). Let (X,Y, Z) be a point chosen uniformly at random 


in the unit cube (0,1)%. Find the probability that the quadratic equation 
Xt? +Yt+Z=0 has two distinct real roots. 


SOLUTION. By the quadratic formula, the quadratic equation Xt? + 
Yt+ Z =0 has two distinct real roots if and only if Y? —4XZ > 0. That 
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is to say, Y > V4XZ. Let A = {(2,y,z): V4az < y < 1,0 < 2,z < 1}. 
Then 
P(Y > 2VXZ) = P((X,Y,Z) € A) 
volume (A) 
volume ((0, 1)?) 


since (X, Y, Z) is uniformly distributed on the unit cube (0, 1)?. The volume 
of A is 


volume (A) = III dx dy dz 
1/(4z) 1/4 
-[ I, a tees a Aes dx dy dz 
z=1/4 Ja 


1/(4z) 
-[ I, (1 — 2\V/xz) java + [ fo (1 — 2\/xz) dx dz 
z=1/4 /a= 











4 . a 
-[ le — V0 3/2) dz +f [1 — 2faz]1_» dz 
1/4 


z=0 


14 1/4 4 
= mut | Gil eae 
1 0 3 


8 3/2)1/4 


| 

| 
o 
In 
+ 

| 


so that 


BO 2X4) = sine 





5 
= 4 2, 
3676 og 


since the volume of the unit cube (0, 1)? is 1. 


PROBLEM (6.9.4). Show that if X and Y are independent random vari- 
ables having the exponential distribution with parameters \ and 1, respec- 
tively, then min{X,Y} has the exponential distribution with parameter 
A+ pL. 
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SOLUTION. The distribution function of Z = min{X,Y} is 


P(Z < z) = P(min{X, Y} < z) 
= 1—P(min{X,Y} > z) 








since min{X,Y} > u 


—_4_p 
a PEP GP ae) if and only if X >u and Y > u 











=1-P(X > z)P(Y > z) by independence 
=1-[1-P(X <2)J[1 PY <2) 

_ f1-(-0)0-0) if z <0, 

= (tS psSQaem)T = G=e-*)]) tee o, 


_ jo if z <0, 
~ 1-2 Ot? if 2 >0, 


so that min{ X,Y} has the exponential distribution with parameter \ + pu. 


PROBLEM (6.9.5). Lack-of-memory property. If X has the exponential 
distribution, show that 


P(X >u+v|X >u) =P(X > v) for u,v > 0. 


This is called the ‘lack of memory’ property, since it says that, if we are 
given that X > u, then the distribution of X — u is the same as the original 
distribution of X. Show that if Y is a positive, continuous random vari- 
able with the lack-of-memory property above, then Y has the exponential 
distribution. 


SOLUTION. The solution to this problem is adapted from the proof of 
Theorem 11.26. 
If X is exponentially distributed with parameter A then, for u,v > 0, 





P(X >u+vand X > u) 
P(X > u) 
P(X >uty) 


(since) = “P(X Su) U < U+tU 





P(X >u+vu|X>u= 











= —___ from Example 5.22 


= P(X So); (6.9.5.1) 
so that X has the lack-of-memory property. 
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Conversely, suppose that X is positive and continuous, and has the 
lack-of-memory property. Let G(w) = P(X > u) for u > 0. The left-hand 
side of (6.9.5.1) is 


P(X >utv|X>u)= oe a oe 


and so G satisfies the ‘functional equation’ 





G(u+v) = G(u)G(v) for u,v > 0. (6.9.5.2) 


The function G(u) is non-increasing in the real variable u, and all non-zero 
non-increasing solutions of (6.9.5.2) are of the form 


G(u) =e" for u> 0, (6.9.5.3) 


where A is some constant. It is an interesting exercise in analysis to derive 
(6.9.5.2) from (6.9.5.2), and we shall check this as follows. First, we note 
that G(0) = 1 by setting v = 0. Next, we differentiate throughout (6.9.5.2) 
with respect to v, noting that G must be differentiable by the continuity of 
X and the Fundamental Theorem of Calculus, to obtain 


G'(u+v) = G(u)G"(y). 


In particular, setting y = 0 gives 


G'(u) = G(u)G"(0). 
Therefore, letting \ = —G’(0), we have 


G'(u) = —AG(u) G(0) =1, 


» 


which has solution G(u) = e~*“ as required. Since G is a decreasing func- 


tion, we must have A > 0. 


PROBLEM (6.9.6). Let X 1, X2,...,Xn be independent random vari- 
ables, each having distribution function F' and density function f. Find 
the distribution function of U and the density functions of U and V, where 
U = min{X, Xo,...,Xn} and V = max{X1, Xo,..., Xn}. Show that the 
joint density function of U and V is 
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fuv(u,r) =n(n—Df(u) fF) - FI"? ifu<v. 
SOLUTION. The distribution function of U = min{X1, Xo,...,Xn} is 


Fy(u) =PU <u) 
= P(min{ X1, X2,..., Xn} <u) 
= 1—P(min{X1, X2,...,Xn} > u) 
since min{X1, X2,...,Xn}>u 
& X1 >u,X2>Uu,...Xn >u 
=1-P(X, > u)P(X2 > u)---P(Xn > u) by independence 
=1-[1- P(X < wf — P(X2 <u] [1 P(X <0] 
=1- [1 F(w)J[l — F@))--- [1 Fw) 
=1-[1- F(u)]”. 


=1-P(X1 >u,X2>u,...Xn > 4) 








Differentiate this equation with respect to u, where possible, to obtain 


fu(u) = + Fo(w 


d 
—~ “4-1 Fu)” 
4p Fwy) 
=nf(u)[l— F(u)]"™. 
The distribution function of V = max{Xj, X2,..., Xn} is 


Fy(v) =P(V < v) 
= P(max{X1, X2,..., Xn} <v) 
since max{X1, Xo,...,Xn}<v 
& XX, <u, Xe <v,..., and Xn <v 
= P(Xy < v)P(X2 < v)---P(Xn < v) by independence 
= F(v)F(v)--- Fv) 
= [F(v)]”. 


= P(X <v,X2 <0,...Xn Sv) 








Differentiate this equation with respect to v, where possible, to obtain 
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For u < v, the joint distribution function of U and V is 


Fu,v (u,v) 
=P(U <u,V <v) 
= P(min{ X1, X2,..., Xn} < u,max{X1, Xo,..., Xn} < v) 
= P(max{X1, X2,..., Xn} <v) 
— P(min{ Xj, Xe,..., Xn} > u, max{Xy, X2,..., Xn} < v) 
min{X 1, X2,...,Xn}>u 


and 
=P(V <v)—-P(u< X1 <u,u< Xo <v,...,u< Xn <v) max{X 1, X2,...,Xn}<v 


eu< XX, <vuu< X2<, 
LeU Xp Sv 
=P(VV <v)—-P(u< X1 < v)P(u< Xo <v)---P(u< Xn < v) 
= [F(v)]” — [F(v) — F(u)|[F(v) — F(u)]--- [F(v) — F(u)] by independence 
= [F(v)|" — [F(v) — Fw)”. 








Differentiate this equation with respect to u, where possible, to obtain 


© Fey(u,v) = 2 {Le (o))" ~ le) — Fw))") 
=nf(u)[F(v) — Fw)", 
and differentiate this equation with respect to v, where possible, to obtain 
oO? FE _ n—2 
Fagg Fe (ts) = nln — DA) F OF) — FOI, 
giving that the joint density function of U and V is 


fu (u,v) = n(n — IF (u) f(w)[F(v) — Fw)" ifu<v, 
as required. 
PROBLEM (6.9.7). Let X1, X2,... be independent, identically distributed, 


continuous random variables. Define N as the index such that 


X, > Xo >-++- > Xn_; and Xy_1 < Xn. 
Prove that P(N = k) = (k —1)/k! and that E(NV) =e. 














SOLUTION. We note that if X and Y are ontinuous random variables, 
then P(X = Y) =0. Henceforth, it is reasonable to assume that 
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X1, > Xo >--+- > Xp_y and Xp_1 < Xp. 


The number of permutations of {X 1, X2,...,X,} is k!. There are k — 1 
cases occur for this event, such as, X, > X, > X29 >--- > Xp_-1, Xi > 
Xp > XQ >-+++ > Xp-1,..., X1 > XQ > +--+ > Xp_-g > X~_ > Xp_1. There 
are k — 1 cases occur for this event. Thus 


as required. 
The mean of N is 




















1 
= S> = reindex with n =k—2 


II 
is 


as required. 


PROBLEM (6.9.8). Show that there exists a constant c such that the 
function 


Cc 
(1 + x2 + y?)3/2 





(aon) = for z,yER 


is a joint density function. Show that both marginal density functions of f 
are the density function of the Cauchy distribution. 


SOLUTION. Joint density functions integrate to 1, so that 
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i=[" f[ tewardy 


CO CO Cc 
= io i Gaetan 


Change variables to polar coordinates in the last integral to obtain 


(=f ee Te aan er as 
Cc 


=2 —sar adr. 
foam’ 


Substitute t = 1+? in the last integral to obtain 


234 





giving that c = 1/(27). 
We now evaluate J = ee ag Cary 572 dy where a > 0. We have that 





— 
| 
Co 
Q 8 

a 

Q 
he 
inne 
Ses lle 
aot) 

Pinal 

bo 





| 
8 

gle 

Q. 

——~ 

as 
w 
Pin 
bo 


| 

: 
= 
Sts 


Substitute t = y/,/a in the last integral to obtain 


rif dt 
A dog (1+ #7)3/2 


Let t = tan@ where —71/2 <  < 7/2, then 1+?#? =1+tan?¢d= ars and 





dt = soa co) dp 
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@ J—n/2 Car 
1 a /2 
= bin dea = 


ie xorg? 
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2 
“(feb Pp] = 
ae a 


Apply the integral above for a = 1+ 2x7, we have that 





eo 
f(a) =f on (1 + x2 + y?)3/2 


1 2 


On 14a? 
i 


~ 71 + 22) 


for x ER, 


so that X has the Cauchy distribution. A similar calculation shows that Y 


has this distribution also. 


PROBLEM (6.9.9). Let X and Y have joint density function 


f(z,y) = ‘3 


i(@+3yje"@) if x,y > 0, 


otherwise. 


Find the marginal density function of Y. Show that P(Y > X) = 2. 


SOLUTION. The marginal density function of Y is 


i= f lag) de 
caw 


= i =(a+ By)e” etY) dx 
o 4 


1 = he 
= qil-(@ + 3y)e met f 2 (a+y) dx} 


x=0 


1 - —(z co 

= qisve Des le : a 
1 

= 7 (bye +e") 
1 as 

= 7 8y +1je"¥ 


for0O<y<o. 


by integration by parts formula with 
u=2+3y,dv =e? dr 
du = dz,v = —e **¥) 
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We have that 


P(Y > X) = [f fen deus 


by Theorem 6.22, where A = {(2,y) € R?: y > x}. Writing in the limits 
of integration, we find that 


PY 2 X)= u if <(x + 3y)e7*) da] dy 
y=0 0 4 


where I is the integral given by 


yo 1 
iS i “(x + 3y)e7@*) da 
«z=0 4 


by integration by parts formula with 
( u= 2+ 3y,dv = et) dx ) 
du = dz,v = —e~ +) 


1 ~(e+y)yy 4 (ety) 
= He + 3M + | et dx} 
xr=—0 


qe + 8ye¥ —e % +e). 


x ple" Bu +1) +6°74(—4y — D] 


Thus 
P(Y > X) =a e 4 (3y +1) +e 74(—4y — 1)] dy 
= lf « ~¥(3y + 1) dy + e 74 (—4y — 1) dy] 
= T(I+K) 


where J and K are the integrals given by 
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fas) e ¥(3y + 1) dy 
0 


= -eHay + ig + f 3e 4% dy 
0 


= [-e-%(3y + 1)]g? + [-3e7]° 
3e Yy —e 4 — 3e 45° 

= [-3e Yy — 4e 415° 

=4 











and 


— e74(—4y — 1) dy 
0 


1 
= [-5e"(—Ay — 1) + eg 


1 
= [2e7*y + eo — e *I° 





3 
= [2e7™y + 5° Ie 


Therefore 


as required. 


by integration by parts formula with 
u=3y+1,dv=e 4% dy 
du = 3dy,v = —e 4 


l ae by integration by parts formula with 
= ee eae — 1° +f 2e~*4 dy u = —4y —1,du = e~ 74 dy 


du = —4dy,v = —5e77y 
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PROBLEM (6.9.10). Let S;, be the sum of n independent, identically 
distributed random variables having the exponential distribution with pa- 
rameter A. Show that S$, has the gamma distribution with parameters n 
and X. 

For given t > 0, show that N; = max{n: S;, < t} has a Poisson distribution. 


SOLUTION. For the first part, we shall prove by induction on n. Con- 
sider first the case n = 2. Then Sy = X; + X29, where Xj and X92 are 
independent, identically distributed random variables having the exponen- 
tial distribution with parameter A, has density function 


fs.(z) = [- fx, (2) fx.(s — x) dx 
fa fx, (a) fx,(s—2)dx ifz>0, 
0 


otherwise, 


since fx, (x) fx,(z — x) =0 unless ¢ > 0 and z—2 > 0. Thus, for z > 0, 


fale) = f Ne eM) de 
0 


= [ Me ** dx 
0 


= [Ne als oy 
= 220", 
1 


~ -7(2) 
The only distribution with density function of the form above is the gamma 
distribution with parameters 2 and A. Hence the result is true for n = 2. 
Let m > 2 and suppose that the result is true for n < m. Then it is 
true that S,, = X1, + Xo +---+ Xm, where Xj, Xo,..., Xm independent, 
identically distributed random variables having the exponential distribution 
with parameter A, has the gamma distribution with parameters m and X. 
That is to say, 


222-1 —rz 


1 m,m—-1l,—-rAz 
oe Tun)? gle if z > 0, 
o 0 if z <0, 
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so that Spry. = Xy+Xo+---+Xm41, where X1, X2,...,Xm4+41 independent, 
identically distributed random variables having the exponential distribution 
with parameter A, has density function 


fa ey= a fom (2) Fxmaa (2 — 2) de 


a fSm(Z)FxXmai(2—2)dx ifz>0, 
0 


otherwise. 


since fs, (@) fXm4i(2—) = 0 unless x > 0 and z— a2 > 0. Thus, for z > 0, 


ai 
FSn1 (2) = / eat eNO) le 
0 


T'(m) 
ae 


= 1 m+1,—Az 
“ita ae 
1 


~ mI'(m) 
1 





intl ym o—rz 


~ T(m +1) 

The only distribution with density function of the form above is the gamma 

distribution with parameters m+ 1 and X. Therefore S,, has the gamma 
distribution with parameters n and 4. 
For the second part, we note that 


mri Merz 


P(M = n) = P(S» <t< Sick) 


= II. Fsessid (O59) dx dy 


by Theorem 6.22, where A = {(z,y) € R?: « < t < y}. Wvriting in the 
limits of integration, we find that 


t co 
PN = n):= / / Fsn,Sin41(@, y) dx dy 
z=0 Jy=t 


t co 
=f of tas. (u | 2)fs4(0) dr dy, 
z=0 Jy=t 


The conditional distribution function of S,41 given S, = wu is 
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P(Sn4i Sy | Sn = 2) =P(Snt+ Xngi Sy | Sn = 2) 
=P(a@+Xn4i <y) 
=P(Xnti Sy — 2). 


We differentiate this with respect to v, to obtain 














FSn4ilSn(Y | 2) = fxnaily - 2). 


Hence 


Fxnyi(y — ©) fs, (#) da dy 


Fxn41(y — ©) fs, (#) da dy 


l| 
ae a 
2555 


t lee) 
= Ne My-x) Ney? 1 e >” dx dy 
he =0 Jy=t ra) 
may f. (Be A(y-2) oe 1 eat dx dy 
«=0 
Ne mie 
ae 


n. 


giving that N; has the Poisson distribution with parameter At. 


PROBLEM (6.9.11). An aeroplane drops medical supplies to two duel- 
lists. With respect to Cartesian coordinates whose origin is at the target 
point, both the x and y coordinates of the landing point of the supplies 
have normal distributions which are independent. These two distributions 
have the same mean 0 and variance o?. Show that the expectation of the 
distance between the landing point and the target is 0\/7/2. What is the 
variance of this distance? (Oxford 1976M) 


SOLUTION. Let X and Y be coordinates of the landing point of the 
supplies. The joint density function of X and Y is 


1 
zo (0? +4"), 





fxy(a,y) = exp( 


Qo? 
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The distance /X?+ Y? between the landing point and the target has ex- 


pectation 


E(V X24 Y?2) = [. i Vur+y?fxy(a,y) dx dy 
1 
=i is Va? + ys 5 exp( ane ty") dx dy 


by Theorem 6.62. Changing variables to et coordinates, we find that 
































1 
f/X24y2 2 2 
B(\/ X24 Y2 X24 Y7?) =f" [om 702° nae )r dr dO 
-saf- fe i eee 
1 
eras 


where J is an integral given by 
ge ps all 

r= | re 202 dr 
0 


by integration by parts formula with 





2 Da 8/0 2 ye fate r2 aero ee) 
= [-r*o“e 207" | +o € 202" dr u=r,dv = re 22" dr 
eee Berey) 
0 du = dr,v = —o7e on 
CO 
=o | V200" dt by the substitution t = 1 i 
? Jat 


2 love) 
ss =f V2c0e7" dt 
2 Je 


—? at 


| 
mm] % 
bo 
to 
wg 8 
ray 


giving that 














242 6. MULTIVARIATE DISTRIBUTIONS AND INDEPENDENCE 


as required. Alternatively, we note that 














lo 
E(\/X?2 + Y?) = od 0 or sae (pear )r dr do 
= do r? p(-—<r’) dr 
Nar cg” Ga Oe 
= a : al see exp( saat) dr since the integrand is even 
< Tv 2 Tv 
22” ~ OV 9? 


as required, since 





a 1 1 
var(X) = / a pa exp( 552 a”) dx = 0”, 
—Co 


where X has the normal distribution with parameters 4 = 0 and o? (see 
Example 5.65). The mean of X? + Y? is 














(x? +7) = f / Ai ure eed 


=f f +P pz ew- sale +9?) drdy 








by Theorem 6.62. Changing variables to polar coordinates, we find that 

















R(X? + Y%) “ho hae 5 exp( 53"?)r dr ab 
o 
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where J is an integral given by 


OS ge di ge 
j= r’e 202 dr 
0 


0 
1 
= / 2o'tet dt by the substitution t = sr" 
66 20 
0 
=e. / te’ dt 
—Cco 
0 by integration by parts formula with 
=So (lei. = i e' dt} u=t,dv=e' dt 
pes du = dt,v = ef 


= 204{0 — (-1)} 
= 201. 


Hence 





1 
R(X? + Y¥?) = 20% 











The variance of VX? + Y2is 


var(V/X? + Y?) = E(X? + ¥*) — [B( VX? + ¥?)? 


























1 


PROBLEM (6.9.12). X and Y are independent random variables nor- 
mally distributed with mean zero and variance o?. Find the expectation 
of /X2+Y%. Find the probabilities of the following events, where a, b, c, 
and a are positive constants such that b< canda< 5m: 

(a) VX2+Y? <a, 

(b) 0<tan7?(Y/X) <aandY >0. 


SOLUTION. The expectation equals o\/7/2 (see Problem 6.9.11). 
(a) The joint density function of X and Y is 





fxy (x, y) = 
The distribution function of WX? + Y? is 
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P(V X2+Y2 <a) =P(X?+Y? <a) 
= ff fv(ou)aeay 


1 1 
= II, Te exp( 552 (a? + y)) da dy 


by Theorem 6.22, where A = {(2,y) € R?: 27+ y? < a}. Changing 
variables to polar coordinates, show that, we find that 








P(/X2+Y2 <a)= [- [hs se 202" dr dO 
6= r= gener 
=20 


= nz a|(-0?)e "Ig 


—s 1 ( 1 

= exp ag2% 
(b) Let R? = X?2+Y? and @ = tan !(Y/X). 
We have that 


P(R? <a,0<0 <a) =/f, fxy (a, y) dx dy 


by Theorem 6.22, where B = {(x,y) € R?: 2?+y? <a and 0 < tan 1(y/z) < 
a}. Changing variables to polar coordinates, show that, we find that 


FR 9(a,a) = ae ie 


1 2 
ne ~ 202° rdrdé 
6= ¢ rao 270? 


[(—07 eae” Jy" 





ax 





== [1 — exp( a)| for a > 0. 


Qa 202 


For a > 0 and 0 < a < 2z, the joint density function of R? and @ is 
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1 1 1 
SiS 6-962 
eae 


Qn 
fr2(@) = fr2,9(a, a) da 
a=0 
2r 
1 i; 1 
= aie 8 Gee 
ae Beare a2 
Lo sty 
=— igo Qo2 


The marginal density function of © is 


CO 
fo(a) = fr2,9(a, a) da 
a=0 
aie eae | 1 
= Sere a ay 
is poe 
- 1 
On’ 


Thus 


fr2,9(a, a) = fr2(a) fo(a) 


giving that R? and © are independent. Moreover, R? has an expenential 
distribution with parameter shy and © has uniform distribution on (0, 27). 
Therefore, 


et a 
P(O < tan” '(Y/X Y>0=/ —da=— 
(0.< tan "(¥/X) <a,¥ >0)= f° Fda = 2 
PROBLEM (6.9.13). The independent random variables X and Y are 
both exponentially distributed with parameter A, that is, each has density 
function 
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—At if 
n= {0 ift > 0, 


0 otherwise. 


(a) Find the (cumulative) distribution and density functions of the ran- 
dom variables 1 — e~** , min{X,Y}, and X —Y. 

(b) Find the probability that max{X,Y} < aX, where a is a real 
constant. 


(Oxford 1982M) 


SOLUTION. (a) The distribution function of Z = 1— e~~* is 





Fz(z) = 











= P(—AX > log(1 — z)) for0<z<1 











Thus Z = 1 — e~*” has a uniform distribution on (0, 1). 
The distribution function of Z = min{ X,Y} is 


P(Z < z) = P(min{X, Y} < z) 
= 1—P(min{X,Y} > z) 





since min{X,Y} > u 
if and only if X >uand Y >u 





=1-P(X >z,Y >2z) 











=1-P(X > z)P(Y > z) by independence 
=1-[1-P(X <2). PY <2) 

_ f1-(G-0)0-0) if z <0, 

iE ae er) a a), 


_ jo ifz <0, 
~ )1l-e?* ifz>0, 
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so that min{ X,Y} has the exponential distribution with parameter 2.. 
Let U = X and V = X — Y. The mapping T of this problem is given 
by T(x, y) = (u,v), where 


U=Z, v=Xr—-Y; 
and T is a bijection from D = {(z,y):2,y € R} to S = {(u,v): u,v € R} 
(or from D = R? to § = R?). It has inverse T~!(u,v) = (a, y), where 
=u, y=u-v. 
The Jacobian of T~! is 
Ox Ox 
ER: 
Ou Ov 


giving by (6.51) that U and V have joint density function 





es 





fuy (u,v) = fxy (xu, v),y(u,v)) |J(u,v)| ifO<u<y, 
; : 0 otherwise, 
7 New¥ New A(u-v) |—1| foO<u< v, 
~ Vo otherwise, 
| eee) On 0; 
ma) Ke otherwise. 
For v > 0, 


fy(v) = a fu,v (u,v) du 


= Ne AGU») dis 


1 
22 yell e 
= on 


je ee?) ee, 
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since |v| =v ifv>0. For v <0, 


fy(v) = in fu,v (u,v) du 


_ | Ne ACGu-v) du 
u=0 


= Fe Neg 


since |v| = —v if v < 0. The marginal density function of V is 


(for v € R,) fv(v) = Rent 


giving that X — Y has the bilateral exponential distribution. 

(b) If a < 0, then max{X,Y} > 0 > aX since aX < 0, so that 
P(max(X,Y)) =0. If0 <a <1, then max{X,Y} > X > aX since X > 0, 
so that P(max(X,Y) < aX) =0. If a > 1, we have that 





P(max{ X,Y} <aX) =P(X <aX,Y < aX) 
=P(Y = @x) 


/ i fxy (0,y) de dy 


by Theorem 6.22, where A = {(2,y) € R?: 0 < 2, y < 00, y < ax}. Writing 
in the limits of integration, we find that 











6. 


© 
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P(max{X,Y} <aX) = i, ert) dex dy 
y 


— )2 ia — feet) ar | da 








y 
=0 
1 1 
—)2 aes —AL) qd 
[yt ye tle 
1 e(at+1) ee 
a+l 
= : +1 
ae os) 
_ a 
eee 


PROBLEM (6.9.14). The independent random variables X and Y are 
normally distributed with mean 0 and variance 1. 

(a) Show that W = 2X —Y is normally distributed, and find its mean 
and variance. 

(b) Find the mean ofZ = X?/(X?2+Y7). 

(c) Find the mean of V/U, where U = max{|X|,|Y|} and V = min{|X]|,|Y]}. 
(Oxford 1985M) 


SOLUTION. (a) We shall find the joint density function of S = X and 
W = 2X —/Y, and then find the marginal density function of W. The 
mapping T of this part is given by T(z, y) = (s, w), where 


S=2, w=2r—y, 


and S is a bijection from D = {(z,y): 2 € R,y € R} toS = {(s,w):s¢€ 
R,w € R}. It has inverse T~!(s,w) = (x,y), where 


L=S, y=2s—w. 


The Jacobian of S~! is 


Ox =O 
ae A ee 
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giving by Jacobian formula, (6.51) that S and W have joint density function 


fs.w(s,w) = fxy(a(s, wv), y(s, w)) |J(s, w)| 





= — 23 — —1|. 
= exp(—5[s? + (28 — w)?)) [-1 
= = exp(—5 (5s? — dow + w2)) 
an exp 5 s sw +w 
The marginal density function of W is 
CO 
f(w) = | fs,w(s, w) ds 
—Co 
29 HAG? te Pi?) 
auf wrk 5 (5s?—-4sw-+w ne 
née 20 
a" peal Te alg 
27 Joo 
ew) ia gr aOR as 
27 at 
1 2 an | 1 
= (w~) — 142 . : 
= —e 276) —e 2° at by substitution t = /5s — —=w 
Qn aay : V5 
—sten(w?) [Pe 
= —e 2r(V5) e 2° dt 
InvV/5 —oo 
(since Te Ie e- 3” dr = ) 
1 2 
= eg a) ag 
InV/5 
= eC. aa (w?) 
2n(V5)? 


giving that W has the normal distribution with mean 0 and variance (V5)? 
(b) The mean of Z = X2/(X?4+ Y) is 
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RX? eve) -[ | = ay wept y) dx dy 


* eo batty?) 
af. i aos on dudy 
1 2 


T foo 7,2 
== an OSE ae 








27 o  # 
1 rn 1,2 
=5,/ cos eae e 2" rdr by Fubini’s theorem 
1 
ee! [seo 0 
Or Jo 2 
ae! ee sna 
DGND ae a 
1 
On 
ak 
==. 
(c) Let T=V/U. Then 
7 maxt X11} 
min{|X| ,|Y|} 


Wl ae (xX) Sq¥1: 


{iF if |X| <|¥I, 
|x| 


The mean of T is 


R(T) = I= Hfxcy avdy + [ff HB fy dedy 


by Theorem 6.62, where A = {(2,y) € R?: |x| < |y|} and B = {(2,y) € 
R?: |x| > |y|}. Interchanging the roles of x and y, we find that 


E(T’) =2 [f 12) oS aihag. 
A ly! 


Writing in the limits of integration, we find that 
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nth [ —3(# +9") do dy 
x£=0 Jy= ov aR 


m/2 0. 
s Lf" [. TcosO 472. a ag by changing variables to polar coordinates 
TS o= 








n/4 9 rsin 0° 
a /2 
= ‘i cose ao [ e232" rdr by Fubini’s theorem 
n/a sind 0 
4p 1 Tn citi vis ' 
— — dt e 2’ rdr by the substitution t = sin @ 
3/2 tbe 





A 2256 
ean 3/2)(—e ?” |o’) 


2 
= — log2. 
T 


PROBLEM (6.9.15). Let X and Y be independent random variables, X 
having the normal distribution with mean 0 and variance 1, and Y having 
the x? distribution with n degrees of freedom. Show that 


xX 


/Y/n 


T= 





has density function 


1 TG ED) ee i ay 
Jin Tn) (14 i. 2 fort ER. 


T is said to have the t-distribution with n degrees of freedom. 


f(t) = 





SOLUTION. We shall find the joint density function of T = X/,/Y/n 
and W = Y, and then find the marginal density function of T. The joint 
dentiy function of X and Y is 


1,2 1 1 . 
Jigen ae ere ECR Dio 
X,Y \4; = 
0 otherwise. 
The mapping M of this problem is given by M(x, y) = (t, w), where 
xo 


it w= YJ, 
vy/n 


t= 
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and M is a bijection from D = {(2,y): 2 E R,0<y<co}toS= {(t,w): 
t €R,0<w < oc}. It has inverse M~'(t, w) = (2, y), where 


/w 
B= hls; y= Ww. 
n 


The Jacobian of T~! is 


0 ) t 
a =| ange | |e 








giving by Jacobian formula, (6.51) that T and W have joint density function 


fxy (x(t, w), y(t, w)) [J (t, w)| ifte R,w > 0, 
0 otherwise, 


1 w,2 1 1 . 
7 cae Tim (2¥) eV ifte R,w>0, 


0 otherwise. 


frw(t,w) = 


The marginal density function of T is 


f(t) = f * Page CaN aie 


gues ee es ee Lo tp aay PW 
= ——e 27" ——__(_yw)2 e 2 —dw 
| V 20 En) 2 ) Von 




















1 1 1lvi 1 ee A 1(74.142 
= 5n-1 5(n+1)-1,-5(1+<t Jw 
Von ny (2) vnJy x 
Substitute v = 5(1 + 41? )w = dv= $(1 =a +4?) dir end ww = oe 
1 1 1.1 1 at 1(7.1,2 
fr Jim 0 (En) 2 Valo © P rm 
1 1 (yi 1 ee) 93 (nt1)—-1,)3(nt1)-1 a 2 P 
= 2 e U 
V2n 20(5n) 2 vn Jo (14 Le2)a@t0-1 1+ie 
1 1 1 1 
= i i ‘i y3rth-le? dy 
Vi Dan) Vr (1 + 242)2D Jo 
(for ¢ € R,) 


_ A TEOHD) 4 Bday 
n 


~ Van Tn) 





since 
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cis 1 
| y2rth)-le-Y dy = T(=(n+1)), 
0 2 
by the definition of the gamma function. 


PROBLEM (6.9.16). Let X and Y be independent random variables with 
the y? distribution, X having m degrees of freedom and Y having n degrees 
of freedom. Show that 








_ X/m 
~ Y/n 
has density function 
T 1 im-1 
f(u) = Oe a . wii? for u > 0. 


U is said to have the F’-distribution with m and n degrees of freedom. 


SOLUTION. We shall find the joint density function of W = X/m and 


U= oe and then find the marginal density function of U. The joint 


dentiy function of X and Y is 





fxy (a, y) = ( 21 (<y) 2" le 2 (@+9) for x, y > 0. 


The mapping T of this problem is given by T(x, y) = (w, u), where 
yee 
y/n 


and T is a bijection from D = {(z,y): 2,y > 0} to S = {(w,u) : w,u > O}. 
It has inverse T~!(w,u) = (x,y), where 


x 
m 


w 
L=muw, y=n—. 


The Jacobian of T~! is 





ge oe m 0 mnw 
Oy ou Se Pine. ab arr Ra, 
dw Ou u ue U 
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giving by Jacobian formula, (6.51) that T and W have joint density function 


fw.u(w, u) = fx,y(a(w,u), y(w, u)) |J(w, w)| 
1 1 tm—1;1) W\in-1 —1L(mwtn¥%) MnwW 
= 2 —n—)2 2 wf 
a(Emyr(en) (2 Gre uw 


The marginal density function of T is 
fo(u) =f fwo(w.u) dw 


oe 1 1 tm—1,l. W\4n-1_-L(mwtn®) MNw 
= mw) 2 =n—)? e 2 u’—— dw 
/ a(im)F(En) 2 ) (5 ae u? 


for w,u > 0 














1 1 Lmtd hmn nym f we lrtn—-1e- 5 (MAS yw dw 
0 


~ ar(am)P(n) 3) 


S 


Substitute v = 5 (EE ay = dv= 5(m +") dw and w = ae 





1 1 ot 1 ieee we: 
= yg (MH), gM, GN _)gntl 
Seam eee 


fe 2uv )2(mtn)—1e-v 2u 
0 mu+n mu+n 








1 1 1 1 1 
5m gryagm-l 


T(5m)0(5n) (mu +n)2r4n) 


os 
8 
koe} 
nl 
3 
ae 
= 
| 
fan 
o 
I 
e 
Q 
2 


4 
3 
Hl 
3 
3 
= 
Nie 
3 
| 
ran 
3 
Nie 





: ts L(m+n)-1,,-0 1 
since v2 e "dv= Ps (m+n)) 
0 


= 
A) 4 
NI- 
3 


| 
a 


NIFOle NI} 





A 
3 
t 
3 





Za 





_ : for u > 0, 
al(Sm)E (sn) [14 (mu/n)] zr 





as required. 


PROBLEM (6.9.17). In a sequence of dependent Bernoulli trials, the 

conditional probability of success at the ith trial, given that all preceding 
trials have resulted in failure, is p; (i = 1,2,...). Give an expression in 
terms of the p; for the probability that the first success occurs at the nth 
trial. 
Suppose that pj = 1/(i +1) and that the time intervals between suc- 
cessive trials are independent random variables, the interval between the 
(n — 1)th and the nth trials being exponentially distributed with density 
n° exp(—n°x), where a is a given constant. Show that the expected time 
to achieve the first success is finite if and only if a > 0. (Oxford 1975F) 
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SOLUTION. First, for 7 = 1,2,..., we define the discrete random variable 
X; by 


y= 1 if the ith trial is a success, 
‘ 0 if the ith trial is a fail, 


giving by the chain rule of probability (also called the general product rule) 
that the probability that the first success occurs at the nth trial is 


P(X, =1,X1 =0,X2 =0, 





















































..,Xn-1 = 0) =P(Xn =1| Xn-1 = Xn-2 =-:: = X1 = 0) 
P(Xn—-1 = Xn-2 =--: = Xi = 0) 
=P(Xn =1 | Xn-1 Xn-2 aay X1 O)P(Xn 1=0|Xn qe: = X 
P(Xy-2 =+:: = Xi =0) 
=P(X, =1| Xn-1 =--: = X1 = 0)P(Xn-1 =0| Xn-2 =-:: = X1 =0) 
P(Xn—2 =0| Xn-3 =-+- = X1 =0)---P(X2 =0| Xi = 0)P(X1 = 0) 





= Pn(1 — pn—1)(1 — pn—2) +++ (1 — p2)(1 — pr), 


since 





PRE S0 | eer )PE S ) 
=P, 





For the second part, since p; = 1/(i + 1), it follows that 














1 1 1 
P(X; =1, Xi = Xi-2 = + Xi =0)= —5 0-5) (0-5) 
- 1 7-1 1-2 2 1 
ae ee ee 
1 
~ (+1) 


Let X be the time to achieve the first success. The mean of X is 
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“a jt it 
ve es “lisp 


1) 1 
ge (G+ 1)t 





If a > 0, then the series }>° converges to some finite number A. Hence 

















jel J 

X)s DUA a 

“an )+Gr tel 
=A. 


Conversely, if the expected time to achieve the first success is finite, 
ay 
that is to say E(X) is finite, — the series )7?° (OL 1 je j=) Gi Fi; COnVerges. 


Thus the general term j=l 7 . mn 
1 
ie j=l je COnverges, implying that a > 0 since the series }*“° 

















is bounded, giving aa the series 


jal 5a converges 
if and only ifa > 0. 
Therefore, the expected time to achieve the first success is finite if and 


only if a> 0. 


PROBLEM (6.9.18). Let a,b > 0. Independent positive random vari- 
ables X and Y have probability densities 


-1- ae 2 
go te, yb ley for x,y > 0, 





respectively, and U and V are defined by 


_ Xx 

— X+Y" 

Prove that U and V are independent, and find their distributions. 
Deduce that 


U=X+Y, 
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(Oxford 1971F) 


SOLUTION. The joint density function of X and Y is 


1 1, l Sayn 
Ixy = Tye te rH! see 


T(a) 
1 1 b-1,—(e+y) 
P(a)P (6) 


(for x,y > 0.) = Cana 
The mapping T of this part is given by T(z, y) = (u,v), where 


x 
g+y’ 





U=X“+Y, v= 


and S is a bijection from D = {(z,y):2,y > 0} toS ={(u,v):0<u< 
oo, 0 < v < 1}. It has inverse T~!(u,v) = (x,y), where 


= Uv, y=u(1—v). 


The Jacobian of S~! is 








O 
ai 8: |= | OF et a ls 
ae (l—v) -wu 


giving by Jacobian formula, (6.51) that U and V have joint density function 


fxy(x(u, v), y(u,v)) |J(u,v)| ifu>Oand0<v<1, 
fu,v(u,v) = 
0 otherwise, 
7 Tare (ur)* “[u —y)otemetel ly ifu>Oand0<v <1, 
10 otherwise, 


oO —vj le ifu>Oand0<v<1, 
0 


otherwise. 
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Since fu,v(u,v) is in the form of g(u)h(v), it follows that U and V are 
independent. The marginal density function of U is 


fu(u) = i. fu,v (u,v) dv 


- ‘e fuv(u,v)dv ifu>0 


0 otherwise, 


since fy,y(u,v) = 0 unless u > 0 and 0 < uv < 1. Thus, for u > 0, 





(by (6.44)) = ——_ uu" “e re 


= Peden). 


The marginal density function of V is 


(by Theorem 6.31) 


fu,v (u,v) 
fv vy) = 
= Fea) 
7 rare eet = v)P-le"T(a a ) aati Taare it 
~ 10 otherwise, 


eda abe <1, 
6 otherwise. 


Since U and V are independent, it follows that 














E(UV) = E(U)E(V). 




















Hence, 
































JE(X +Y), 
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giving that 






































R(X )\ = (EX +Y). 
Thus 
xX K(X) E(X) 







































































E = = 
(¥ re E(X+Y) E(X)+E(Y)’ 
as required. 
PROBLEM (6.9.19). Let X,, X2, X3 be independent y? random vari- 
ables with r1, ra, r3 degrees of freedom. 
(a) Show that Y; = X,/X2 and Y2 = X; + X2 are independent and 


that Yo is a NG random variable with r; + rg degrees of freedom. 
(b) Deduce that the following random variables are independent: 


X1/T1 a X3/T3 
Xo/rT2 (X1 + X2)/(ri + 12) 


(Oxford 1982F) 








SOLUTION. (a) The joint density function of X; and X2 is 


aig oe pt dies Se i 
fx,,X2(@,y) = oe oF ae ea Hay > 0, 
1,442 ? 


0 otherwise, 








1 1)4(ri4re)—2,,471-1, drg—1,—43 (ety); 
= { So ret y? e ? if z,y > 0, 


0 otherwise. 
The mapping T of this part is given by T(x, y) = (u,v), where 
x 


us, vearty, 
¥y 


and S is a bijection from D = {(z,y):2,y > 0} to S = {(u,v): u,v > O}. 
It has inverse T~!(u,v) = (x,y), where 
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The Jacobian of S~! is 








Ox Ox oe 1 
UL gv = (u+1) utl | _ = z + is u = is 
x Ot Gent “at (u+1)? ut l (w+l1)? utl (ut+1)?’ 


giving by Jacobian formula, (6.51) that Y; and Y2 have joint density func- 
tion 


J(u,v)| if u,v > 0, 


otherwise, 


fX1,X2 (a(u, v), y(u, v) 
0 


fy v2 (u, v) = 


(4) 3 i tr2)-2( ue ygri 1 wy are—te” a (arer tat) 
uw uw 











1 % ; 
= {PETS (utl)2 ifu,v > 0, 


0 otherwise, 





(4) 3 @tr2)—2y arr -1( 1 )2(ratra)ag (ritra)—1e— 30 if u,v > 0, 


1 aero Sa 
= {res utl 


0 otherwise, 


Since fy,,y,(u,v) is in the form of g(u)h(v), it follows that Y; and Y2 are 
independent. The marginal density function of Yj is 


fy, (u) = [- fu,v (u, v) du 


= dae fuv(u,v)dv ifu>0 
~ (0 otherwise, 


since fy, y,(u,v) = 0 unless u,v > 0. Thus, for v > 0, 
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fy, (u) 
= he 1 (2) 30r1tr2)-2 
0 40 (4ri)P(4r2) 2 
u2ri—l/ 1 )z(ritr2)a3(ritr2)—1e— 3% dy 
util 
= 1 (2) 3 (ra tr2)-2y 371-1, 1 ya (mitra) 
40(4ri)P(4r2) 2 u+1 
OO 4 1 
| pe ritr2)-le~ 3” dy 
0 
= i 1S (ritr2)-29,371-1/ u y2(ritra) 
40(4ri)P(4r2) 2 utl 
co 
1 
ip (2t)2(r1tr2)-le-t ge by the substitution t= 59 
0 
1 Lidia sigs corey ay wh Ae 1 = 
= ya (ritra)—24271—1/ a (ritr2)g3(ri+r2) 
40 (4ri)0 (412) 2 util 
Oo 9 
| po (ritra)—-1le-t gp 
0 
1 1 A(ry+re)—-2, dry-1 1 L(rytre)g4(ry+re)—-1 
= ja ritr2)—24, 371-1 jaritragg (titre 
4r(4ri)0(4r2) 2 util 
ri +72 
rote) by (5.41) 
rytr: 
a7 Ns") 3r—1( i ya(ritra), 
21 (4r1)P(4r2) util 


The marginal density function of Y2 is 
(by Theorem 6.31) 


fro (v) = fyiye (u, v) 


fy, (u) 
ty d(ritre)-2,, 471-171) 4 (ri4r2),, 4 (ritr2)-1,-40 
array ra) (2) me (ga)? v2 e2"2 
T(sriy(sr sacl B i a ey ee 
=} RANG dn -D( ay det en 
my) 
° otherwise, 
1 1,,\4(ritre)-1,-4u; 
_ | ward 2")? e3” ifu>0, 
0 otherwise, 


so that Yo isa x? random variable with r; + rg degrees of freedom. 
(b) We note that if X and Y are are independent, then g(X) and h(Y) 
are independent for all functions g,h: R > R.Let 
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X rgX,— Xy/r1 
= at d h(x X95) = 
AX ry Xq  Xe/r2 on eee) 


X3/7r3 
(X1 + Xo)/(ri + 12)” 








Since X,/X»_ and X, + X92 are independent, then 


X1/r1 ar X3/T3 
X9/r2 (X1 + X2)/(r1 +12) 








are independent. 


PROBLEM (6.9.20). Let X and Y be random variables with the vector 
(X,Y) uniformly distributed on the region R = {(4,y):0< y <a < l}. 
Write down the joint probability density function of (X,Y). Find P(X + 
Y +1). 

Find the probability density function fx(x) of X, and also find E(X). 
Find the conditional probability density function fy;x(y | x) of Y given 
that X =, and also find E(Y | X = x). (Oxford 2005) 


























SOLUTION. The joint density function of X and Y is 


fxy(a,y) = il if (x,y) ER 


0 otherwise. 


Let A= RN{(z,y) € R?: 2+y <1}. Then 


pUx+y <i)= ff 2ardy 


l| 
oo 
ll wl 
° 
so 
ll re 
< | 

Rd 

bo 

Q 

8 

Q 

Neng 
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The probability density function of X is 
fx(x) = i x,y (x,y) dy 


Sa if0<a<l 


0 otherwise. 


‘ if0<ar<1 


0 otherwise. 


The mean of X is 

















The conditional probability density function of Y given that X = x is 


fyxty |) = Sew 


x if0<y<a<1, 
0 otherwise, 


_ 1 PO<y Srl, 
0 otherwise. 














ve od 
E(Y |X) = — 
(|x) =f ycdy 


= 2 for0O<a<l. 


PROBLEM (6.9.21). Let X and Y be independent random variables, 
each uniformly distributed on [0, 1]. Let U = min{ X,Y} and V = max{X,Y}. 
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Show that E(U) = 5, and hence find the covariance of U and V. (Cam- 
bridge 2007) 











SOLUTION. The joint distribution function of U and V is 


Fu,v (u,v) = PU <u,V < v) = P(min{ X,Y} < u,max{X,Y} < v) 
= P(max{X,Y} < v) 
— P(min{X, Y} > u, max{X,Y} < v) 
=P(X <v,Y <v) 


since min{X,Y}>wu 
-Pu<X <v,u<Y<v) and max{X,Y}<v 
eucx<vuu<Y<u 


= Fx y(v,v) —-P(u< X <v)P(u< Y <v) | by independence 


=v? —(v—u)’, 
for0<u<vu< 1, since 
1 if0<a<1 
0 otherwise, 


and 


Oe i ap 0 pt 
oa 0 otherwise. 
The joint density function of U and V is 


0? Fyy(u, v) 
fuv (u,v) = Fae. 


=2if0<u<v<il. 


The marginal density function of U is 


fu(u) = [2x 


= 2), 


= 2(1 — u) if 0<u< 1. 
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The mean of U is 

















The covariane of U and V is 


cov(U,V) = 


The mean of UV is 









































































































































Le spl 
E(UV) = i) / 2uv du dv 
u=0 Ju=u 


etna 
=| 2u=V" |=, du 
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The marginal density function of V is 


The mean of V is 


43/1 
3° lo 
aie 
at 
The covariance of U and V is 
De, alee 
U,V) =--=--= 
cov(U, V) ieee ee 
ae: 
36 


PROBLEM (*6.9.22). Three crew members of Dr Who’s spacecraft Tardis 
are teleported to the surface of the spherical planet Zog. Their positions 
X,Y, Z are independent and uniformly distributed on the surface. Find the 
probability density function of the angle¢ XCY, where C' is the centre of 
Zog. Two people positioned on the surface at A and B are in direct radio 
communication if and only if ACB < 51. 

(a) Find the probability that Z is in direct radio communication with 
either X or Y, conditional on the event that @:= <XCY satisfies 6 < 5m. 

b) Find the probability that Z is in direct radio communication with 
both X and Y, conditional on the event that ¢ > 51. 

Deduce that the probability that all three crew members can keep in 
touch is (a + 2)/(47). 


PROBLEM (6.9.24). Let X and Y be independent non-negative ran- 
dom variables with densities f and g, respectively. Find the joint density 
function of U = X and V = X +aY, where a is a positive constant. 

Let X and Y be independent and exponentially distributed random vari- 
ables, each with density 


(for x > 0.) f(x) =r*** 


268 6. MULTIVARIATE DISTRIBUTIONS AND INDEPENDENCE 


Find the density of X + 5Y. Is it the same as the density of max{ X,Y}? 
(Cambridge 2007) 


SOLUTION. The joint density function of X and Y is 


(for ty 0) fxy(2,y) = f(x)g(y). 
The mapping T of this problem is given by T(z, y) = (u,v), where 
uU=, v=“ ay, 


and T is a bijection from D = {(z,y):2,y > 0} to S = {(u,v) : u,v => Of. 
It has inverse T~!(u,v) = (x,y), where 





v-wU 
CS; y= : 
a 
The Jacobian of S~! is 
0 fe) 
ff B-LAge 
Bu dv “a a a 


giving by Jacobian formula, (6.51) that U and V have joint density function 


fxy (eu, v),y(u, v)) |J(u,v)| ifu,v 2 0, 
0 otherwise, 


fu,v (u,v) = 
2 oe if u,v > 0, 


0 otherwise, 


Apply the above result for the sepcial case where f(x) = g(x) = Ae~*” for 
x>Oanda= 5 to obtain, 


fetus Dre’ ew *—-4) if OSU < v, 
ae 0 otherwise, 
7 Q2eAQZ—-4) iF O<uK< v, 
~ 10 otherwise. 


The marginal density function of V = X + sY is 
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fv(v) =} Deen NOY) aa 
0 


2.-2r ee 
= —2Xru Vv 
= 2)“e —|u=0 


(for v > 0,) =e (et 1). 


Let Z = max{ X,Y}. The distribution function of Z is 





Fz(z) = 

















(by independence) =P 


The density function of Z is 
0 if z <0, 
fale) = eer —e™) ifz>0, 


giving that the density of X + 5Y is not the same as the density of 
max{X,Y}. 


SOLUTION (6.9.25). Let X and Y have the bivariate normal density 
function 


2 








f(x,y) 2pxy + y")} for a,y ER. 


1 1 
= ex x 
Qn/1 — pe pt 2b By! 
for fixed p € (—1,1). Let Z = (Y — pX)/\/1— p?. Show that X and Z 


are independent N(0,1) variables. Hence or otherwise determine P(X > 
0,Y > 0). (Cambridge 2008) 


SOLUTION. For the first part, we shall use the Jacobian method to find 
the joint pdf of W = X and Z. The mapping T of this part is given by 
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T(x, y) = (w, z), where 
y — pr 
era 
and T is a bijection from D = {(z,y):2,y € R} to S = {(w, z) : w,z € R}. 
It has inverse T~!(w, z) = (x,y), where 


Cw, y=pwt+vV/1- pz. 


The Jacobian of S~! 

ox 1 0 
oy Bs | = | 

Oz 


Bw Dalle 
‘WwW 


giving by Jacobian formula, (6.51) that W and Z have joint density function 


w=, Z= 

















_ 1 1 
LIND Sai age ae) 
1 1 
= 95 EL Bap) 
( for w,z € R.) 
1 Deon ig 
Se OH oan) 
The marginal density function of W is 


=| fw,z(w, z) dz 
. [om eo 3 w742") ge, 


= Ano 1,2 
“hen 








tw 
= gl 


7 
L. 
Qa 


(for w € R,) = age 


Von 


so that W has the normal distribution with mean 0 and variance 1. By 
symmetry, the random variable Z has this distribution also, and 


(for u,v € R,) fw,z(w,z) = fw(w) fz(z) 


so that U and V are independent. 
For the second part, we have that 


[w? — 2pw(pw + V/1 — p22) 4 t /1 = p2z)7]}/1 — p? 
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P(X >0,Y > 0) = ff fxy (a, y) dx dy 


by Theorem 6.22, where A = {(a,y) € R?: x,y > 0}. Writing in the limits 
of integration, we find that 


P(X >0,¥ >0)= f i 1 —_<e aap Pe iy 
a=0 Jy=0 2ty/1 — p? 


(by changing variables to polar coordinates) 


1 ie ie = 1 (r? —r? p sin 26) 
= = e 2(1=p?) r dr d0 
2n/1 — p? Jo=o Jr=0 






































= 1 2 3 ea psin 20)r? l-p Pe odo 
27/1 — p? Jo=o0 1 — psin2¢ 
eo 
ae 
(by the cbancaeh 
aE. 
dt t=tand 
(1 + t?)( Fo) 
a? 
dt 

f° =m 

ee: 
dt 

kG FeUSe ) 
1- 1 7 1 
a — al aa 

9 Vi=p2 
/1— p? —1 t—p co 
on 3V1—p? tan ae 

: (ec tan~! ——? ) 
2m 2 1-2 


Alternatively, we have that 
P(X >0,Y >0) =P(W >0,pW + V/1- p?Z > 0) 
= / fw,z(w, z) dw dz 
A 


by Theorem 6.22, where A = {(w,z) € R?: w > 0,2 > —pw//1— p?}. 
Writing in the limits of integration, we find that 
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CO (oe) 1 
P(X >0,Y >0)= / fl e242) dy dz 


w=0 a PU 21 


(by changing variables to polar coordinates) 


1 2 ve ee: 
=— e 2’ rdrdé 
20 é=tan-1 ae 5 /r=0 











De 5) an ae = 2 is 
PROBLEM (6.9.26). Let X and Y be random variables with the joint 
probability density function 


1 
(for x,y > 0.) fxyv(z,y) = cose 


Show that the joint probability density function of U = 3(X — Y) and 
V=Yi is 


se" if (u,v) EA 


0 otherwise, 


fULV (uo) = 


where A is a region of the (u,v) plane to be determined. Deduce that U 
has probability density function 


() ee sel aoe nebe: 


(Oxford 2008) 


SOLUTION. The mapping T of this problem is given by T(x, y) = (u,v), 
where 


1 
u=5(z— 4), Cy 
and T is a bijection from D = {(z,y):2,y > 0} to A= {(u,v): 2u+u> 
0,v > O}. It has inverse T~!(u,v) = (x,y), where 


xr=2ut+, yY=v. 
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The Jacobian of S~! is 


fe) 0. 
JE El=[o il? 
re 0 1 


giving by Jacobian formula, (6.51) that U and V have joint density function 


1¢~2[Qutv)+u]9 if (u,v) € A 


0 otherwise, 


fu,v (u, v) = 


- on if (u,v) EA 


0 otherwise. 


The marginal density function of U is 


fu(u) = ‘a fu,v (u,v) dv 


_ Joo Het du ifu>0O 
= [oume dv ifu<0O 
~Y ifu>0 
ifu<0O 


(—0oo < u < 00,) =-e ™, 


II 
Nle -_e_ 
NIF NIFH 
ay) oO 
e 


as required. 


PROBLEM (6.9.27). (a) Suppose that the continuous random variables 
X and Y are independent with probability density functions f and g, both 
of which are symmetric about zero. 
(i) Find the joint probability density function of (U,V), where 
U =X and Vi= Vix. 
(ii) Show that the marginal density function of V is 


fy(v) = 2 fo xf (x)g(xv) dx. 


(iii) Let X and Y be independent normal random variables, each 
with mean 0, and with non-zero variances a? and b?, respectively. Show 
that V = Y/X has probability density function 
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fv(v) = eh 


where c = b/a. Hence find P(|Y| < |X). 

(b) Now let X and Y be independent random variables, each uniformly 
distributed on the interval (0,1). By considering the random variables 
U =Y and V = XY”, or otherwise, find the probability density function 
of V. 

(Oxford 2010) 


for —co <u<o, 


SOLUTION. (a) For part (i), the mapping T of this part is given by 
T(x,y) = (u,v), where 
= _Y 
U=2, v= 
x 
and T is a bijection from D = {(z,y) € R?} to S = {(u,v) € R?: u 4 Of. 
It has inverse T~!(u,v) = (x,y), where 


L=u, y = uv. 


The Jacobian of S~! is 


1 0 


Uv U 


Ox Ox 
| - = U, 


U 
y oy 
Ou Ov 
giving by Jacobian formula, (6.51) that U and V have joint density function 


f(u)g(uv) |u| if (u,v) € S$ 


0 otherwise. 











fuv(u, v) = 


For part (ii), the marginal density function of V is 


(since the integrand is even) = [ uf (x)g(uv) du 
0 


as required. For the first part of part (iii), we shall use the Jacobian method 
with U = X and V = Y/X. The density functions of X and Y are 
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and 


(599) 

= —— exp(—~3y’*), 

Vv 27nb2 me Ope 

both of which are symmetric about zero. It follows that we may calculate 


the marginal density function of V by applying the results of part (i) and 
(ii) to the random variable V to obtain 








I 

i) 
et. 
ae 
Ke 

. 
ce S| 
8 
bo 
ra 

a 
Sain 

8 

e 

Q 

8 


(& 
= g(gzt 
mab Wp 


(by the ieee 
be 2 
Loli 20 
-t 2 
= dt t=(— 
=a 0 coe aa 
=F ee 
~ rab BHazy2’ © /=0 
ab 
m(b? + a2v?) 
ab 
a2 
(Ete?) 








m((3)? + v?] 
a for —c<vu< 0M, 
n(c? + v?) 


where c = b/a, as required. For the second part of part (iii), we have that 


P(|Y| < |X) = ff fxy (a, y) dx dy 
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by Theorem 6.22, where A = {(x,y) € R?: |y| < |x|}. Writing in the limits 
of integration, we find that 








P(Y| < |X|) =4 / / ge) at Gedy 

















«=0 Jy=0 V 27a? 2b? 
Alternatively, we have that 
Y 
P(|Y X|)=PUs| <1 
(IY] <|XI) = PC S| < Y) 
P(|V| < 1) 
P(-l1 << V <1) 
(by Theorem 5.27) = ‘p fv (v) dv 
-1 











= tan = 4 ea 
=I 
= —(tan7'— —tan™! =) 
2 1 
(since tan7!(—a) = — tan~! a) == tan = 
7 C 


(b) Since X and Y are independent, the joint density function of X and 
Y is 


fxy(z,y) = fx(x)fy(y) 


jb awe0<ay<1 
~ 10. otherwise. 
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The mapping T of this part is given by T(x, y) = (u,v), where 
U=Y,; v= oy" 


and T is a bijection from D = {(2,y) € R?:0<2,y <1} to S={(u,v) € 
R?:0<u<1,0<v<vu’}. It has inverse T~1(u, v) = (x,y), where 





v 
LS a? y=u 
The Jacobian of T~! is 
Ox Ox 20 1 1 
Ou Ov 
giving by Jacobian formula, (6.51) that U and V have joint density function 


1 ‘ 
fuy (u,v) = i if (u,v) € S$ 


0 otherwise. 


The marginal density of V is 


fv(v) = i fu,v (u,v) du 


Sa if0<v<1l, 


0 otherwise, 
Lj : 
Be eres HO <od 
0 otherwise, 


Vo 


0 otherwise. 


ae if0<v<l, 


PROBLEM (6.9.28). (a) Define the distribution function F’ of a random 
variable, and also its density function f, assuming F is differentiable. Show 
that 


d 
= -—P(X : 
f(a) = -LP(X > 2) 
(b) Let U, V be independent random variables, each with the uniform 


distribution on [0, 1]. Show that 
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1 2 
BV SU Se) = 38+ 40 for x € (0,1). 


(c) What is the probability that the random quadratic equation x2? + 
2Va+U =0 has real roots? 

(d) Given that the two roots Rj, Rz of the above quadratic are real, 
what is the probability that both |Ri| < 1 and |Rg| < 1? 


SOLUTION. (a) The function Fy: R — [0,1] is called the distribution 
function of a random variable X if and only if 
Faj=PXx <2); 

The non-negative function fx is called the density function of a random X 


if and only if the distribution function Fy (x) of X may be written in the 
form 


F(a) =P(X <a)= f f(x) du forz ER. 


for some , we say that X has (probability) density function (or pdf) f. 
We have that 


(since F is differentiable) f(x) = —F(za) 


as required. 

(b) Since U and V are independent random variables, each with the 
uniform distribution on [0,1], it follows that the joint density function of 
U and V is 
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fuv (u,v) = fu(u) fv (v) 


2 jb a ae et 
~ 10. otherwise. 


We have that 


P(V?>U>2)= II. fxy (x,y) dx dy 


by Theorem 6.22, where A = {(u,v) € R?:2 <u <1,Vu <v < Ih. 
Writing in the limits of integration, we find that 


il 1 
PV?>U>a)= f i du dv 
u=x Jv=/u 
1 


=f a-vajau 
=u— <u3/|} 
1 2 
= goer ae for x € (0,1). 


(c) By the quadratic formula, the random quadratic equation 2?+2V 2+ 
U =0 has real roots if and only if V? —U > 0. That is to say, V? > U. 
We have that 


P(V? >U) =P(V? >U > 0) 
1 
ne 
by applying the result of part (b) with x = 0. 
(d) Wrong Solution: Given that the two roots Ri, Re of the above 
quadratic are real, both |Ri| < 1 and |R2| < 1 if and only if the derivative 


of the quadratic to be negative at —1 and positive at 1. That is to say, 
VES 0. 
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P(|Ri| < 1,|Re| <1,V? > U > 0) 

P(V? >U >0) 

P(—2 + 2V <0,2+2V >0,V?>U>0) 

P(V? >U >0) 

P(VV <1,V>-1,V?>U>0) 
P(V? >U >0) 

PVA Let SU 0) 

P(V2 > US 0) 

PVs SU > 0) 

P(V2 >U > 0) 

=1. 


P(|Ri| <1,|Ro] <1|V?>U>0)= 
































Right Solution: Note that the quadratic equation x? + 2Vx2+U =0 
has two real roots R1, Re such that both |R,| < 1 and |Re| < 1 if and only 
if 


which is to say that 


VU 
U-2V+1>0 
U+2V+1>0 
—l<V<i1 


where f(x) = 2? +2Vx+U. We have that 


P(V? 2 U,U -2V +130,U+2V4130,-1<Ven= |f fu,y (u,v) dudv 
A 


by Theorem 6.22, where A = {(u,v) € R?:0 <u<l,Jfu<vu< “'}. 
Writing in the limits of integration, we find that 
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1 url 
PV? 2 U,U-2V +120,U+2V4130,-1<Ven)=/ [- du dv 
u=0 J Ju 


= [CE - vou 





2 
We te.» ° Dea 
ra era 
ail 
=5. 


The probability that both |R,| < 1 and |Rg| < 1 given that the two roots 
R,, Re of the quadratic are real is 





P(|Ral < 1,|Reo| <1,V? >U) 

P(V? >U) 
P(V? >U,U-2V 4+1>0,U+2V4+1>0,-1<V <1) 
P(V2 >U) 








P(|Ri| < 1,|Ro| <1| V7 >U) = 

















CHAPTER 7 


Moments, and moment generating functions 


7.1. A general note 
7.2. Moments 
EXERCISE (7.10). If X is uniformly distributed on (a,b), show that 





(for he 1235) E(X*) = 


























b 
(by Theorem 5.58) E(X*) = , ae — dz 
_ 1 b+I)b 
(b—a)(kK+1) : 
pktl _ gktl 
OER near) 6 —alk+1 


as required. 


EXERCISE (7.11). If X has the gamma distribution with parameters w 
and X, show that 














(for k= 1, 2,35) E(X*) = 


SOLUTION. Since X has the gamma distribution with parameters w 
and X, it follows that 
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(by Theorem 5.58) BOX) = 7. oa NW g@—1e—A2 dp 

= a i NW yg ktw-l Ae ada 

= Ew i. wk pw+k-1 rx dp 
(fork Oy suai) =e 


as required. 


EXERCISE (7.12). If X has the y? distribution with n degrees of free- 
dom, show that 











(for k= 1,9) 03..) E(X*) = 2" 





SOLUTION. Since X has the y? distribution with n degrees of freedom, 
it follows that 




















k = ze 1 1 in-1,-iz 
(by Theorem 5.58) LX y= es mere a) CM aT 
Co 
_— : / Gr ght3 n— Linge ay 
oe n) 
ae LS =yk+3 npktgn—1o—52 ae 
T'(5n) 
T(k Ly 
(fork = 1.9,.<;,) = 2k ste) 
(57) 


as required. 


7.3. Variance and covariance 


EXERCISE (7.35). If X and Y have the bivariate normal distribution 
with parameters /41, [/2, 01, 02, p (see (6.76)), show that 


cov(X,Y) = po 102 and p(X,Y) =p 
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SOLUTION. Since X and Y have the bivariate normal distribution with 
parameters (41, 2, 01, 72, p, it follows that 


(by (7.21)) 
































cov(X, Y) =E(XY) —E(X)E(Y) 


EXERCISE (7.36). Let X1, X2,... be a sequence of uncorrelated random 
variables, each having variance o?. If Sn = X1 + Xo+---+ Xn, show that 


cov(Sin, Sn) = var(Sm) = mo? if, We 


SOLUTION. By (7.21), 


























cov(Sm, Sn) = E(SmS,) — E(Sm)E(Sn) 


















































ifxj i=1 i=1 
4=1,2,...,m 
gS Dike nN 








































































































= mvar(X1), 


since X 1, X9,... are a sequence are uncorrelated random variables, each 
having variance o?. 


EXERCISE (7.37). Show that cov(X,Y) = 1 in the case when X and Y 
have joint density function 


E(X;X;) — E(X,)E(X;)] 
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0 otherwise. 


fe,y) = aa ME TOD, 


SOLUTION. By (7.21), 








cov(X, Y) = E(XY) — E(X)E(Y). 


























The mean of X may be calculated directly from (5.59) in the usual way. 
This calculation is a little complicated since the marginal density function 
X is not easy to calculate, and it is simpler to use the following trick. 
Solving E(Y) first. The marginal density function of Y is 














_ od Lewy dx if y>0, 
0 otherwise, 


_ ee ify > 0, 


0 otherwise, 
|e? aba: 
0 otherwise. 


The mean of Y is 














The conditional expectation of X given Y = y is 
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l| 
San 
8 
e£/8 
“| 
e| 
Q 
8 


. = : by integration by parts formula with 
= —{[-rye ¥]254+ i ye ¥ dx} ui=t,dv=e 1 dz 
: du =dz,v=—ye ¥ 


l| 
oo 
L 8 
~ 
e118 
Q. 
8 


























=[-ye ¥] 20 
The mean of X is 
L(x) = i E(X | Y = y)fy(y) dy by Theorem 6.69 
0 


lI 
as 
8 
end 
aa 
e 
Q. 
Ned 














The mean of XY is 
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(oe) co 1 dtd 
E(XY) -| i. xry—e * » dxdy 
«2=0 Jy= 
=0 [. 
= i. (f ze oy dx) dy 
ye xz=0 


Hence 
































as required. 
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7.4. Moment generating functions 


EXERCISE (7.59). Find the moment generating function of a random 
variable having 

(a) the gamma distribution with parameters w and 4, 

(b) the Poisson distribution with parameter X. 


SOLUTION. (a) If X has the gamma distribution with parameters w 
and A, the moment generating function of X is 


[o-e) 
1 
Mx(t) =| ee Ne ide 
0 





(w) 
= a i ete" dy 
7 ia / ne i tet dis be the Ribs aS SDS 
= Toyacae wea 
z rar) ace 7 * gle-* de = T(w) 
= coy ift <X. 


(b) If X has the Poisson distribution with parameter \, the moment 
generating function of X is 





Sarr | 
Mx(t) = eae 
k=0 . 
oo t)k 
es 3 (e*) 
=e 
k! 
k=0 
CO Ok 
_ A Aet : > ze op 
=e “e since a € 
k=0 
.: t 
—e A+tAe ; 


EXERCISE (7.60). If X has the normal distribution with mean pw and 
variance o, find E(X°%). 


SOLUTION. Since X has the normal distribution with mean p and vari- 
ance a”, the moment generating function of X is 














290 7. MOMENTS, AND MOMENT GENERATING FUNCTIONS 





My(t)= [eee eee g 

eh) = e ae 20 a 
—oo 

—2@=p)" _ ore) 

26 dx 





1 e3 2 2 
G2 (a*—2ya+p*—2to“x) ax 














oa 1 1 7.2 2 2\2 2pto2—t?2.04 

= eo deze —2(ut to" )at++(u+to") : lide 
—co V210 
oe 1 2 21424 

= e302 [z—(p-+to~)] e202 (2uto“+t*o*) ae 








1 242-4 ad 1 1 pe, pp 2y2 
egaz (ute? +20 ul ov abe be (4t0?)P ge 
—Co 


Note the completion of the square in the exponent. The function within the 
final integral is the density function of the normal distribution with mean 
y+ to? and variance o?, and therefore this final integral equals 1, giving 
that 


‘lf 
M(t) Sex Chee") 


pttto7t? 


The derivatives of Mx(t) are 


d 

ag x) =(u+ op ehtt sre 

xv 

2 
; yMx(t) = oreltt art? 4 (4 ot)2ehttaeh — [a2 + (4+ ot) Jet ar? 
XL 

d3 24) 2 ptt 16242 ; Ss 3 ee 
qua x(t) = Au + o*torel $5077 1 24 (ut ot)2|(u + 2b) eMt ar, 


Hence 
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d 
B(X°) = qs Mx leo 
= Quo? + (07 +p?) 
= (0? + p? + 207) 
= i + 3007. 














EXERCISE (7.61). Show that, if X has a normal distribution, then so 
does aX + b, for any a,b € R with a £ 0. You may use Theorem 7.55 
together with (7.51) and (7.58). 


SOLUTION. Let X be a random variable having the normal distribution 
with parameters ps and o?. The moment generating function of aX + b is 


Max +0(t) = 
= et i etn 1h e722 PH)” day 
—C 


fore) 

_ tb 1 1 {{a—(ut+o? at]?—2u0? at—0*a?t?} 

e€ —— € 20 dx 
—co V2n0? 


| 
i 
8 
ae 
e 
8 
+ 
2 
e 
ei 
s+ 
a 
= 
me 
Q 
8 








ti jiapete2g2e? 2 1 ~L e—(uto2at)|? 
a HT ee Ei € 202 du 
—co V 270? 





= elbtpat+ $0? a*t? 


= elanto)tt+5 (ac)?t? 


since the integrand in the latter integral is the density function of the 
normal distribution with mean p+ o7at and variance a”, and thus has 
integral 1. The only distribution with moment generating function of the 
form above is the normal distribution with mean a+b and variance (ac). 
We deduce that aX +6 has this distribution by appealing to Theorem 7.55. 
Alternatively, we consider the linear function aX +6 of the random variable 


X. Ifa,bER, 


Mx(t) = elit gore 


and 
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M.x p(t) = ate + a) 

_ E(e%* et) 

=e E(e\*)*) by (6.63) 
= e!’ Mx (at) 


= etbeu(at)t+ 50° (at)? 





























= elotap)t+ s (ac)? t? 


which we recognize by (7.58) as the moment generating function of the 
normal distribution with mean ay + b and variance (ac)*. We deduce that 
ax + b has this distribution by appealing to Theorem 7.55. 


EXERCISE (7.62). Let X 1, Xo,... be identically distributed}] random 
variables with common moment generating function WM. Let N be a ran- 
dom variable taking non-negative integer values with probability generating 
function G, and suppose WN is independent of the sequence (X;). Show that 
the random sum S = X,+ X9+---+Xy has moment generating function 
Ms(t) = G(M(t)). 


SOLUTION. The moment generating function of S' is 


lErrata from solution writer: be independent, identically distributed 
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Ms(t) = E(e’*) 

















I 

= 
—~ 

o 
ch 
M4 


4 Xi) 





B(et Lins Xi [LV = hPOV =k) by Theorem 2.42 











Me 











= S\(E(ee E14) "P(N = k) 























= EE 1%) HP = k) 


since 








E(et Dia Xi) = E(e'*1 ef X2 ewig efXr) 












































giving that 


7.5. Two inequalities 


EXERCISE (7.72). Determine which distributions on the non-negative 
reals, if any, with mean p are such that 2u is a median. 


SOLUTION. Let X be a random variable. The real number m™ is called 
a median of X if 


294 7. MOMENTS, AND MOMENT GENERATING FUNCTIONS 


POX ie) SS 


Nl rR 


Exercise 5.14 was to show that every random variable possesses at least one 
median m. Since X is non-negative, it follows that 














“PX Syn) 5 Ss 
m m 


by Markov’s inequality. Therefore, any median m satisfies m < 241. Here we 
shall determine whether or not equality can hold. Let A = {w EQ: X(w) > 
m}. The equality can hold if and only if P(X > m) = § and A= Q\{w € 
Q: X(w) = 0}. Thus 


Nile 


P(X <m)=1-P(X >m) 
1 
=5: 
Consider the discrete distribution X that places probability 5 on the each 
value 0 and 24. Then E(X) = yu and m = 2p, as required. 














EXERCISE (7.73). Let J be an interval of the real line, and let f: J ~ R 
be twice differentiable with f(a) > 0 for x € I. Show that f is convex on 
I. 


SOLUTION. By Taylor’s theorem, 


f(t) = f(v0) + f'(ao0)(t — to) + 





Hence 


f(t) = f(xo) + f'(xo)(t — x0) 
since f”(€) > 0 for € € J. Thus 


f(ao) < f(t) — f'(wo)(t — a0). (1) 
Taking v9 = ax + (1-—ay) and t =z for a € [0,1] and z,y € J. Then 


flax + (1—a)y) < f(x) — fi(ax+ (1—ay))(1-a)(@—y). (2) 
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Taking v9 = ax+(1—a) andt=y. Then (1) becomes 


flar+(1—a)y) < f(y) + fi(ar+(l—-a)yja(x—y). (3) 
Multiply throughout (2) by a, multiply throughout (3) by 1 — a and add 
together to obtain 


flax + (1—a)y) < af(a) + (1 — a) f(y). 


Therefore f is convex on I. 


























EXERCISE (7.74). Show by Jensen’s inequality that E(X?) > E(X)?. 


SOLUTION. The function g(x) = x? is convex on the interval (—oo, oo). 


By Jensen’s inequality applied to a random variable X with finite mean, 

















E(9(X)) = E(X”) 



























































> g(E(X)) 
= [E(X))’, 
so that 
R(X) (XY 
EXERCISE (7.75). The harmonic mean 7 of the positive reals 71, %2,..., Un 
is given by 


UT 


Tet tal 3 1 
ney ti 
Show that 7 is no greater than the geometric mean of the 2;. 


SOLUTION. By Arithmetic/geometric mean inequality applied to posi- 


. 11 1 
tive reals er epee gS 
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‘a | 
i Pea 
= 1 1 
Sie ia 
ey xr rn 
1 


Thus 


< W721 %2Q°+ + Ly. 
Therefore 7 is no greater than the geometric mean of the 2. 
7.6. Characteristic functions 


EXERCISE (7.95). Show that the characteristic function of a random 
variable having the binomial distribution with parameters n and p is 


g(t) = (q+ pe)”, 
where g = 1—p. 


SOLUTION. If X having the binomial distribution with parameters n 
and p, then 


k=0 
= (q+ pe)” by binomial expansion 
where q=1-—p. 
EXERCISE (7.96). Let X be uniformly distributed on (a,b). Show that 
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If X is uniformly distributed on (—b, b), show that 


1. 
ox(t) = jpg Sin 


SOLUTION. Since X is uniformly distributed on (a,b), it follows that 





2 Spee SS — tr B 

=litb—oay® 
eitd iz, eita 

~ it(b— a)’ 


as required. By applying the result above to X being uniformly distributed 
on (—b, b), we obtain 


eith _ e ith 


ox(t) = it) 
7 (cos(tb) + 2 sin(tb) — (cos(tb) — i sin(tb)) 
2itb 





_ 2isin(tb) 
- Qitb 

1 
= bt sin bt, 


as required. 


EXERCISE (7.97). Find the characteristic function of a random variable 
having 

(a) the gamma distribution with parameters w and 4, 

(b) the Poisson distribution with parameter 4X. 


SOLUTION. Since X has the gamma distribution with parameters w 
and X, it follows that 
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= ao | gu-le-OA-it)x dz 


T(w) Jo 
at PO [ . aa ae : du by the subsititution u = (A — it)x 
T(w) Jy ‘\—it (\ — it) sf = 
Nw 1 











neg OS pwn lenu mM 
Toes |, 
BN nai 

T'(w) (A — it)” 


Sicne X has the Poisson distribution with parameter A, it follows that 





_ 1 
ox(t) = So etth— —X 
k! 
k=0 
ie os (Aelt)* 
Se kl 
k=0. 
me) k 
at z 
—Agre since S- il =e* 
k=0 
a at 
e A+Ae 


EXERCISE (7.98). If X and Y are independent and identically dis- 
tributed random variables, show that 


$x-y(t) = |¢x(@)I’. 


SOLUTION. We have that 
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= x(t) since X and Y are identically distributed random variables. 


Note that X and Y are independent, so are X and —Y. By independence 
and Theorem 7.87, 





as required. 


7.7. Problems 


PROBLEM (7.7.1). Let X and Y be random variables with equal vari- 
ance. Show that U = X —Y and V = X +YY are uncorrelated. Give an 
example to show that U and V need not be independent even if, further, 
X and Y are independent. 


SOLUTION. We have that 





















































































































































cov(U, V) = E(UV) — E(U)E(V) 
= E((X —Y)(X +Y)) —E(X —Y)E(X +Y) 
= E(X? — Y*) — (E(X) - E(Y))(E(X) + E(Y)) 
= E(X”) — E(Y*) — (E(X))’ + (E(Y))? 


= 0, 


so that U and V are uncorrelated. 
Let X and Y be independent random variables, each being uniformly 
distributed on (—1,1). Then the chracteristic function of U = X —Y is 
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and the chracteristic function of V = X + Y is 


ox+y (t) = [bx(t)]? 
= (7 sin t)? 
5 sin? t. 


Thus U and V have the same distribution by Thereom 7.88. Therefore 


du+v(t) = d2ox(t) 





= px (2t) 

__ sin(2t) 

ae 

Fa sin” t a sin” t 
= gu(t)ov(t), 


giving that U and V are dependent. 


PROBLEM (7.7.2). Let X1,X2,... be uncorrelated random variables, 
each having mean p and variance o?. If X = n7!(X, + Xo4+---+ Xn), 
show that 


1 
n—-1 

















E( S0(%i — X)?) = 0?. 


i=1 
This fact is of importance in statistics and is used when estimating the 
population variance from knowledge of a random sample. 


SOLUTION. We have that 
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ig =a 1 a aan 
spe Gee atest Cece) 
i=l i=1 
1 ” 
- E(S (X? -2XiX +X 
n-— 1 Ou V tu + )), 
1 = a 
= —E()) X - 2X 0 Xi + nX") 
=| 4 
1 a 3 3 et 
= EQ X? — 2nX +nX°) since — 5) X; = X 
a = 
1) ' 5 
= —— (DC E(X?) — 2nE(X") + nE(X")) 
i=1 
eer ; 
= 5 EX?) ~ nE(X")). 
41 
Note that 
n n 
4(X?) = )“Ivar(X;) + (E(Xi))?| 
i=1 i=1 
=n(o" +p”) by symmetry, 


and 
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i=1 
1 n 
Fe aS 
=e << Gy 
4=1,2,...,n—1 
1 n 
Sar (X?) +2 > E(X;X;) 
ae ae 
aa he ee ee 
1 n 
= 5 E(X2) +2 S> E(X;)E(X,;) by uncorrelatedness 
n 
ame Ne 
PH heya 1 
-_ 2 2 (n—1)n 4 
=—4lo" +p) +2 ee 
(n~ 1) 
= —(o7 + p*) + Ve 
Thus 
yt Se 1 1 (n—1) 
z xX, -X)2) = ae 24 2) 4 os veh 
(ag = YP) = tro? + 2) — nl? 2) + 
eee O42 L Din 19 n (n—1) 4 
pa lO re 


as required. 


PROBLEM (7.7.3). Let X,,X2,... be identically distributed, indepen- 
dent random variables and let S;,, = X; + X9+---+X,. Show that 














E(—) = — for m <n, 


provided that all the necessary expectations exist. Is the same true if 
we Sn! 
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SOLUTION. For m <n, 


























































































































n(2m) = oa 
oi ae S, 
eRe Lan arD apm 
xX 
= mE( =~) by symmetry. 
Note that 
Sn 
1 = E(— 
(3) 
Xj 
=n (ys 
giving that 
a. G eee 
() = 
Hence 
a(S) = “ form <n, 


as required. The result is generally false when m > n. See also Problem 
3.6.8. 


PROBLEM (7.7.4). Show that every distribution function has only a 
countable set of points of discontinuity. 


SOLUTION. We prove that {x: F(x) — limyts, F(y) > 1/n} < n for all 
n by contradiction. Assume that there is some positive integer n such that 
there are m points 71 < @ <-+: < %m satisfying F(x) —limy, F(y) > 1/n 
and m > n. Then there exists y1, y2,---, Ym such that F(a;)— F(y1) > 1/n 
and 2-1 < y < 2;. The fact that F'(x;) — F(y;) > 1/n is equivalent to the 
fact that P(X € (y,2i)) > 1/n. Hence 
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which contradicts the fact that P(A) < 1 for any event A. Therefore, 
{v: F(a) — limyte F(y) > 1/n} <n for all n, so that every distribution 
function has only a countable set of points of discontinuity 


PROBLEM (7.7.5). Let X and Y be independent random variables, X 
having the gamma distribution with parameters s and 4, and Y having 
the gamma distribution with parameters t and A. Use moment generating 
functions to show that X + Y has the gamma distribution with parameters 
s+tand X. 


SOLUTION. By Theorem 7.52, 


Mx+y(t) = Mx(t)My (t) 
Xr r 


Sea) aa) 


= Ga if t< A, 


which we recognize by Exercise 7.59(a) as the moment generating function 
of the gamma distribution with parameters s +t and . We deduce that 
X +Y has this distribution by appealing to Theorem 7.55. 


PROBLEM (7.7.6). Let X1, X2,..., X, be independent random variables 
with the exponential distribution, parameter A. Show that X, + Xg+---+ 
Xp has the gamma distribution with parameters n and X. 


PROBLEM. By Theorem 7.52, the sum S = X, +---+ Xy of n inde- 
pendent random variables has moment generating function 


Ms(t) = Mx, (t)Mx, (t)--- Mx, (¢) 
fA)” ift<A, 
~ )oo ift>.. 


The only distribution with density function of the form above is the gamma 
distribution with parameters n and 4. 


PROBLEM (7.7.7). Show from the result of Problem 7.7.5 that the y? 
distribution with n degrees of freedom has moment generating function 


M(t) = (1 —2t)-3" ift< - 
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Deduce that, if X 1, X2,...,X, are independent random variables having 
the normal distribution with mean 0 and variance 1, then 


Z=XP4+ XP +---4+X? 


has the y? distribution with n degrees of freedom. Hence or otherwise 
show that the sum of two independent random variables, having the x? 
distribution with m and n degrees of freedom, respectively, has the y? 
distribution with m+n degrees of freedom. 


SOLUTION. For the first part, we note that if X has the gamma distri- 
bution with parameters w and A, then the moment generating function of 
X is 


Mx(t) = (57)", 


and note that the y? distribution is the same as the gamma distribution 


with parameters w = $n and \ = 5: It follows that the y? distribution 


with n degrees of freedom has moment generating function 





BS 
Cs 
wa 
I 
“——~ 
Nilr 
~" 
Nie 
3 


“— 
SS 
Nie 


"(aye 


™(S)-a"(1 — 24)" 


SS 
Nile 


II 
— 
| NO tll NON Ml eel Ot 


1 
= (1—2t)72" ift<5. 


For the middle part, we shall find the moment generating function of X? 
and use Theorem 7.52. The moment generating function of X? is 
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co V2T 
= 1 —3(1-2t)a? go 
66 V2 


5 U 


Or AL 1,2 1 
= ——e 2" ——_— qu by the substitution u = V1 — 2tx 
te V2 V1 — 2t ss 





1 i. Li. 271553 
= e 2" du 
eS DEY) eee a On 
1 
ry ey 
it 
= (1—2t)-3 ift <5. 


The only distribution with moment generating function of the form above 
is the x? distribution with 1 degrees of freedom. By Theorem 7.52, the sum 
Z = X24 X3+---+ X? of n independent random variables has moment 
generating function 


Mz(t) = Mx2My2---Mx2 
1 
= (1—2¢)~2" ift<>5. 


The only distribution with density function of the form above is the y? 
distribution with n degrees of freedom. 


PROBLEM (7.7.8). Let X1, X2,... be independent, identically distributed 
random variables and let N be a random variable which takes values in the 
positive integers and is independent of the X;. Find the moment generating 
function of S = X; + X9+---+ Xy in terms of the moment generating 
functions of N and X , when these exist. 


SOLUTION. We use the partition theorem, Theorem 2.42, with the 
events B, = {N =n} to find that 
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= 3 B(e(X1+-+Xw) | N =n)P(N =n) _ by Theorem 2.42 


























= S0(Mx, (t))"PWW = 2) by (7.54) 
= Gn(Mx, (t)) 
by the definition of Gy. 


PROBLEM (7.7.9). Random variables X1, X2,..., Xj have zero expec- 
tations, and 











El Xin An) = One for m,n =1,2,...,N. 





Calculate the variance of the random variable 


N 
Z= Ss" GG. 
n=1 


and deduce that the symmetric matrix V = (umn) is non-negative definite. 
It is desired to find an N x N matrix A such that the random variables 


N 
a ee forn = 1,2,...,N 
pal 


are uncorrelated and have unit variance. Show that this will be the case if 
and only if 


AVA’ =I, 


and show that A can be chosen to satisfy this equation if and only if V 
is non-singular. (Any standard results from matrix theory may, if clearly 
stated, be used without proof. A’ denotes the transpose of A.) (Oxford 
1971F). 
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SOLUTION. For the first part, we have that 


























N 
E(Z) =E()) an Xn) 
n=1 


N 
= )/aE(Xn) 






























































n=1 
=0 
since random variables X1, X2,...,X~ have zero expectations, and 
N 
E(Z”) =E(()_ anXn)”) 
n=1 
N 
= S > afE(X;) +2 S- On QmE(XnXm) 
= n<m 
l<n<n-l1 
N 
= Ss azu2, +2 Ss AnAmUnm- 
ra n<m 


























Hence 
var(Z) = E(2Z?) — (E(Z))? 
N 
= Ss" dou +2 S- AnAmUnm- 
n=! n<m 
l<n<n-l1l 
Let 
V1i1-—U12 VIN 
U21 V22 +++) (VAN 
V = ) 


UN1 UN2 **' UNN 
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a=(a a2 ‘'* an ie 
Then 
Vil U12 VIN ay 
; Val Vaq +++ VAN a2 
Va= 
UN1 UN2 °*** UNN an 
ye ne 
j=] ViVi 
N 
ae AjV2i 
= . ’ 
pal eee 
j=1 MiUNi 
so that 
N 
pe AyVii 
Soy ai v04 
j= 1 UiV2i 
aVa =( a ag +: an ) —<, 
N 
ini UNI 
= ) A, Vij; 
1<1,j<N 
= var(Z) > 0 
Thus aVa’ > 0 for all the vector a = ( aj a2 -*: Gan yi so that the 


symmetric matrix V = (umn) is non-negative definite. 
For the second part, let 


aii a2 Sit Qin 
agi a22 °°: GaN 
A = S ’ 
aN1 aNn2 aNnN 
X=(X, Xp Xn ) 


Then 
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ayy 42 ain X 
ag} a2 +++ aN X92 
AX’ = 
QN1 @N2 °°: G@NN XN 
N 
ypae.6 
N 
yp pao 
N 
4 Oa 
Y 
Yo 
Yn 











Suppose that Y;’s are uncorrelated and have unit variance. Since E(Y;) = 0 
for all i, it follows that E(X;Y;) = 0 for i 4 j and E(Y,?) = 1. By the first 
part, 









































1 0 0 
0 1 0 
0 0 1 


where J is the identity matrix. But 
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N N 
E(YiY;) = EO) ain X1)(S~ ajrXk)) 
t=1 k=1 











= Ss" QitAjk H(X;X,) 
1<t,.k<N 


= y AitAjkVUtk 


1<t,k<N 





Syst 
=aVa,, 


where @; = ( Ail aj -*: Qin ae Note that aiVa;, is the element at the 
ith row and jth column of AV A’. Therefore AVA’ = W = I. 
Conversely, if AVA’ = I, then 


1 ifi=j, 
0 ifi4y, 














(¥¥)) = 








van(¥i) = E(¥?) ~ (B(%))? 




















and 
































E(Y;Y;) = E(Y%i)E(¥;) 
= 0, 
for 7 ~ 7. Thus Y;’s are are uncorrelated and have unit variance. 


PROBLEM (7.7.10). Prove that if X = X,+---+X, and Y = Y,+---+ 
Yn, where X; and Y; are independent whenever i # j, then cov(X,Y) = 
Soy, cov(X;, Y;). (Assume that all series involved are absolutely conver- 
gent.) 
Two players A and B play a series of independent games. The probability 
that A wins any particular game is p*, that B wins is q?, and that the game 
is a draw is 2pq, where p+ q = 1. The winner of a game scores 2 points, 
the loser none; if a game is drawn, each player scores 1 point. Let X and 
Y be the number of points scored by A and B, respectively, in a series of 
n games. Prove that cov(X,Y) = —2npq. (Oxford 1982M) 
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SOLUTION. For the first part, we have that 


Note that 














and 


B(XY) 





cov(X,Y) = 








E(XY) — 























1si,jgn 


z(X,Y;) 














1si,jgn 





= EK) +0 


= BUG) +O 




















Hence 




















































































































E(X)E(Y). 











Wares ede sa) 
=E( > Xj) 


E(X;)E(Y;) by independence 









































= E(X, +--+ Xn)E(Yi +--+ + Yn) 
=E()) X)JEQ Y)) 
i=l j=l 
= OF E(Xi))(>> E(Y;)) by independence 
i=l j=l 
= ST E(X EY) + E(XDE(Y) 
i=1 iAj 
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cov(X,¥) =} E(XiYi) + SO E(XJE(¥;) — OL E(%) 
i=1 Aj i=1 
= SC E(XY¥) — EXE) 
i=1 i=1 














= S-(E(X:%) - E(X)E(%)) 




















n 
= > cov(Xj, Xe) 
i=1 


as required. 
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For the second part, let X; and Y; be the number of points scored 
by A and B, respectively, in ith game. Then X = X,+---+ X, and 
Y =Y,+---+ Yn, where X; and Y; are independent whenever i # j. By 


the first part, 


n 
cov(X,¥) = $° cov(X;, Yi). 
i=1 











In order to find cov(X;, Y;), we should calculate E(X;), 








E(Y;) and 





We have that 


pif X; = 2, 
P(X;) = 4 2pq_ if X; = 1, 
0 if X;=0, 
q? if X; = 2, 
P(Y;) = 4 2p¢ if X; = 1, 


1—q?-2pq if X;=0, 


and 











£(X;Y)). 
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0 if X,Y; = 2, 
P(XiY;) = 4 P(X; = ¥1) if XY; = 1, 
0 if X;Y; = 2, 
= < 2nq if X;Y; = 1, 
2 = 9" —4pg if KGY, = 0. 
0 if X,Y; = 2, 
= < 2nq if X,Y; = 1, 
1—2pq if X;Y; = 0. 
Hence 
E(X;) = 2p” + 2pq 
= 2p, 
E(Yi) = 2q° + 2pq 
= 2q(¢ + p) 
= 2q, 
E(XiYi) = 2pq, 
cov( Xi, Yi) = E(Xi Yi) — E(X;)E(¥;) 
= 2pq — Apq 
= —2pgq, 
so that 


n 
cov(X,Y) = S| cov(X;, Yi) = —2npq. 
i=1 


PROBLEM (7.7.11). The joint moment generating function of two ran- 
dom variables X and Y is defined to be the function M(s,t) of two real 
variables defined by 
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M(s,t) = E&**+¥) 











for all values of s and t¢ for which this expectation exists. Show that the 
joint moment generating function of a pair of random variables having the 
standard bivariate normal distribution (6.73) is 


1 
M(s,t) = exp[, (s° + 2pst + t?)]. 


Deduce the joint moment generating function of a pair of random variables 
having the bivariate normal distribution (6.76) with parameters (1, M2, 01, 
02, P- 


SOLUTION. For the first part, we shall first find the joint density func- 
tion of U = X and V = sX +tY by the method of change of variables, and 
then find the marginal density function of V. 

We consider first the case t £ 0. The mapping T of this case is given 
by T(z, y) = (u,v), where 


ua, v=sx+ty, 


and T is a bijection from D = {(z,y): x,y € R} to S = {(u,v) : u,v € R}. 
It has inverse T~!(u,v) = (x,y), where 


_ _ 8 {2 
L=Uu, y= fu au 
The Jacobian of T~! is 
0. 0. 
oa ae ea ees 
OU O04) "| ae ae 
Ou Ov t ob 








giving by Jacobian formula, (6.51) that U and V have joint density function 


fu,v (u, v) = fxy(x(u, v), y(u, v)) | J(u, v)| 
one 





55 [u?—2pu(—futtv)+(—sut40)?| 


1 
On lt|/1— p? 
1 
2r |t| \/1 — p? 


2 
sy [14299453 )u2— 3 (ptts)uvt bv?) 





1 
e 2(1-p 
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For m > 0, then 


Aa? yew God SPAN ieee Ee 
: enBlmw+fut9) gy = , eal ePt+9-£ a, 


—~oo —oo 


2 oo 
= elem f a grat dt 





The marginal density function fo V is 


CO 
fv(v) =| fu,v (u, v) du 
—Co 
1 v2 A(pt-+s)2 
= 1 vom e aah le wasapg By” 
Qn |t| /1— p? vs?+2est+t? 
|t]\/1—p? 





| 





























- ! oa pry Sa 
V/2r1/s2 + 2pst +t? 
1 1 ? (1~p?)t? a 
= e 2(=p?)t? t2+2pst+s2 
V2r1/s? + 2pst +? 
1 1 


= v2 
e 2(s24+2pst+t2) 








7 /21(s2 + 2pst + t?) 


so that V = sX +tY, where t £0, has the normal distribution with mean 
0 and variance s? + 2pst + t?. Thu 


















































E(e®*t+t¥) _ E(eV ) 
oo 1 1 2 

- / ev e 2(s2+2pst+t2) © dv 

v=—co \/277(s? + pst + t?) 
—_ 1 a Jo aaa v?—2v) dv 

Jf 2n(s? + 2pst + t?) Jy=—oo 

1 4(s2+2 st-4 2) 
_ Jane? + Opst a BS) Vin\/s2 + 2pst + Peal ad 
T(s° + Z2pst + 





= 4 (s?+2pst+t?) 
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We consider next the case t = 0. The moment generating function of V is 


My (é) = E(e**$) 
B(e(8)* ) 

















= Mx(sé), 





where 


giving that 
My (é) = 62°, 


which we recognize by (7.58) as the moment generating function of the 
normal distribution with mean 0 and variance s?. The marginal density 
function of V is 





so that 

















Therefore, in both case, the moment generating function of the pair (X,Y) 
is 
M(s,t) = eb s20stee), 


For the second part, let the pair (X,Y) have the bivariate normal den- 
sity function of (6.76), and let U and V be given by (6.78). Then U and V 
have the standard bivariate normal distribution. By the result of the first 
part, the moment generating function of the pair (U,V) is 


Muv(s, t) = M(s, t) 


es (s?+2pst+t?) 
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Thus 





Mx.y(s,t) = E(eXt) 


= B(e(1U tea) +Ho2V +H2)) 




















_ eshittya B (e271 Heavy 











= eHittne Mu,v (801, to2) 


= eH +2 Mf (801, to2). 





PROBLEM (*7.7.12). Let X and Y be independent random variables, 
each having mean 0, variance 1, and finite moment generating function 
M(t). If X +Y and X —Y are independent, show that 


M(2t) = M(t)?M(-t) 


and deduce that X and Y have the normal distribution with mean 0 and 
variance 1. 


SOLUTION (1). For the first part, note that X and Y are indpendence, 
so are X and —Y. We have that 


M(2t) = Mx (2t) 


= Mox(t) by (7.51) 

= Mx+y)4(x-yv) (4) 

= Mx+y(t)Mx_y(t) by independence and Theorem 7.52 

= Mx(t)My(t)Mx(t)M_y(t) by independence and Theorem 7.52 

= M?(t)M_y(t) since X and Y have the same finite mgf M(t) 
= M?(t)My(-t) by (7.51) 

= M(t)>M(-t), 


as required. 
For the last part, let 7(t) = M(t)/M(-t). Then 
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vet) = 7 
_ MOM (1 
~ MO 





= v(t). 
Thus w(2t) = w?(t). This implies that 


for all t. Repeat the procedure to obtain 


v(t) = (We) 


for alln EN. Ast =0 and n = 1, then 7(0) = (~(0))?. This implies that 


w(0) =0 or w(O) 1: 
But w(t) > 0 for all t, it follows that (0) = 1. We note that 


(0) — MMO + M'(O)M(0) 





#2 
v(t) = ¥0) - Sv") 
i MW 


But 


320 7. MOMENTS, AND MOMENT GENERATING FUNCTIONS 





vi) = Toy] =v GQeG) 
vi" = Sw Gl = 50" Ge) + 5UG, 


giving that 


w"(0) = 5"(0) 
since y’(0) = 0. Hence 


y"(0) =0., 
so that (t) = 1 + o(t?) as t > 0. It follows from 


v(t) = (Wa) 
that 
YO =W 
= Jim V5) 
=0 


for alln € N. Thus w(t) = 0, but w(0) = 1, so W(t) = 1. Therefore, M(t) = 
M(-t) since ~(t) = M(t)/M(-t). The equation M(2t) = M%(t)M(-t) 


becomes 


M(t) = (M(5))*. (1) 
Let h(t) = In M(t). We deduce from (1) that 


In M(t) = 4in M(5), 
giving that 


Let 


and 


for some a. Hence 


so that 


But 
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g(t) = A) 


t 
4h(5) 
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M"(0) = E(X?) 
=a 














since E(X) = 0 and var(X) = 1. And, M’(t) = 2ae*” + (2at)2e%”. So 
2a = 1, giving that a = 1/2, and hence 











_ M*(t)M(-t) 
~~ M3(-t)M(t) 
= M(t) ? 
M(-t) 
= v(t). 
Thus w(2t) = ¥?(t). This implies that 





v(t) = Wa)” 


for alln € N. Ast =0 and n = 1, then (0) = (~(0))?. This implies that 





w(0) =0 or a0): =, 
But w(t) > 0 for all t, it follows that (0) = 1. We note that 
; M'(0)M(0 + M’(0)M(0) 
0 = 
oe (MOP 





= 2E(X) 
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By Taylor’s theorem, we have 


But 





giving that 


since w’(0) = 0. Hence 


y"(0) =0., 
so that (t) = 1+ 0(t?) as t > 0. It follows from 


Y(t) = Wa)” 
that 
YO) =W Gg) 
= Jim YH" (5) 


=0 
for alln € N. Thus y(t) = 0, but (0) = 1, so H(t) = 1. Therefore, M(t) = 
M(-—t) since 7(t) = M(t)/M(-t). The equation M(2t) = M3(t)M(—t) 
becomes 
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giving that 


M(t) = (M(55))"" 
Note that 
M(t) = M(0) +M'(0)-t+ 5M"(0) 2 + aM". 
But 








M(0)=1, M’(0)=E(X)=0, M"(0) = E(X?) = var(X) =1. 




















giving that 








Hence 





noo Qn 
1 Coss ae 
= taal + 5° gw + OCga)) 


PROBLEM (7.7.13). Let X have moment generating function M(t). 

(a) Show that M(t)M(-t) is the moment generating function of X—Y, 
where Y is independent of X but has the same distribution. 

(b) In a similar way, describe random variables which have moment 
generating functions 


1 a aT 
I—M(’ : M(ut)e~“ du. 


7.7. PROBLEMS 325 


SOLUTION. For part (a), note that X and Y are independent, so are X 
and —Y. We have that 


Mx_y(t) = Mx(t)M_y(t) by independence and Theorem 7.52 
= Mx(t)Mv(-t) by (751) 
= M(t)M(-t), 


since X and Y have the same moment generating function M(t) as they 
have the same distribution by appealing to Theorem 7.55. 

For the first part of part (b), let Xo, X1, X2,... be independent, iden- 
tically distributed random variables and let N be a random variable which 
takes values in the non-negative integers and is independent of the X;. 
Then the moment generating function of S = X9 + X, + X9+---+ Xy 
is Ms(t) = G(M(t)) (see Problem 7.7.8), where G(s) is the probability 
generating function of N. Consider 


We have that 





1 cil 
(lit Git oot *") 


Nl MLE NleR 


k=0 


giving that G(t) is the probability generating function of the random vari- 
able N that having the mass function 


1 


Pk ~ ok+T fork =0,1,2,.... 


Thus the moment generating function of S is 
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For the last part of part (b), Let Y be the random variable that has 
the density function fy(y) = e~” for z > 0, that is, Y has the exponential 
distribution with parameter \ = 1. Thus 
































Therefore the function Gs(t) = {5° M(ut)e du is the moment generating 
function of U = XY. 


PROBLEM (7.7.14). Coupon-collecting problem. There are c different 
types of coupon, and each coupon obtained is equally likely to be any one 
of the c types. Find the moment generating function of the total number 
N of coupons which you must collect in order to obtain a complete set. 


SOLUTION. Let Y; for «7 = 1,2,...,¢ — 1 be the additional number of 
coupons collected, after obtaining 7 distinct types, before a new type is 
collected. Then each Y; has the geometric distribution with parameter 
(c — t)/c (see Problem 3.6.12), and moment generating function of Y; is 

















ct CM: 
My,(t) = Ye ( — Sty 
k=1 
S64 oS ik Ltke a? 
ae Cc Cc 
k=1 
4 os (ies 
Co Se 
— % _ iet 
c 
(c—i)et 
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It is not difficult to show that the moment generating function of Yo = 1, 
My,(t) = E(e*°) = e', also satisfies the formula given above, so long as one 
allows i = 0. Since Y; are independent, we conclude from (7.54) that the 
total number N = Yo + Yj +--- + X-_1 of coupons which we must collect 
in order to obtain a complete set has moment generating function 














Mn (t) = Myo (t) My, (t)--- My,_, (¢). 
_ I (c— i)et 
AL c_iet ° 
i=0 
PROBLEM (7.7.15). Prove that if ¢, and ¢2 are characteristic functions, 
then so is 6 = ag, + (1 — a)d¢2 for any a € R satisfying O< a <1. 


SOLUTION. If ¢;(t) and ¢2(t) are the chractereistic function of the dis- 
crete random variables X and Y with the mass funtions py = P(X = k) 
and qx, = P(Y =k), respectively. Then 


CO 
agy(t) + (1 — a)d2(t) = 455 e*kp, + (l—o ys Cau 
k=0 k=0 


(oe) 
= Soe lap, + (1 - a)aul- 
k=0 
Let Z be the discrete random variable with the mass function s, = 
P(Z =k) =apyt+(1—a)qgy for k = 0,1,2,... (ap, + (1 — a)qx € [0,1] and 
yo p-olope + (1 — a)qx] = 1). Then adi (t) + (1 — a)do(t) = po ese = 
E(e"*) = oz(t), where z(t) is the chracteristic function of Z. Hence 
adi (t) + (1 — a)do(t) is a characteristic function. 
If ¢(t) and @2(t) are the chractereistic function of the continuous ran- 
dom variables X and Y with the density funtions fx(x) and fy(y), respec- 
tively. Then 

















(oe) (oe) 


e'® fy (x) dx + (1— a) | e' fy (x) dx 


—oo 


ANE A ey a | 


-{*" e"®lafx(x) + (1—a)fy(x)] dx 


Let Z be the continuous random variable with the density function fz(x) 


afx(x) + (1—a)fy(2) (afx(x) + (1— a) fy(2) 2 0 and f™ [a fx(2) + 
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E(e'4) = z(t), where ¢z(t) is the chracteristic function of Z. Hence 
adi (t) + (1 — a)do(t) is a characteristic function. 


(1 — a) f(a] dx = 1). Then a¢y(t) + (1 — a) a(t) = fe" fala) de = 














PROBLEM (7.7.16). Show that X and —X have the same distribution 
if and only if dx is a purely real-valued function. 


SOLUTION. X and —X have the same distribution if and only if X and 
—X have the same chracteristic function by appealing to Theorem 7.88. 
This is equivalent to 


x(t) = ox(-t) 
(et) (e a) 


i(costX + isintX) = E(cos(—tX) + isin(—txX)) 















































since cos(—tx) = costX 
and sin(—tX) = —sintX 








2iE(sin(tX)) = 0 

















tio 


(sin tX) = 0. 
This is equivalent to ¢x(t) is a purely real-valued function. 
PROBLEM (7.7.17). Find the characteristic function of a random vari- 


able with density function 


cae se for x ER. 


PROBLEM (7.7.18). Let X 1, X2,... be independent random variables 
each having the Cauchy distribution, and let 


1 
An = (hit Xa to + Xn). 
Show that A, has the Cauchy distribution regardless of the value of n. 


SOLUTION. Each X; has the Cauchy distribution with characteristic 
function 


ox(t) = e!4 fort ER. 


Since Y; are independent, we conclude from (7.54) that By, = X1 + X2+ 
--- + X, has moment generating function 
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bs, (t) = $x, (t)Ox,(t)--- dx, (t) 


giving that A, = 4(Xy +X94+---+ Xp) = +B, has moment generating 
function 


ba, (t) = ?1p,, (t) 


1 
= bp,(-1 by (7.51) 
= eo" nt 
=e ll, 
which we recognize by Exercise 7.80 as the characteristic function of the 


Cauchy distribution. We deduce that A,, has this distribution by appealing 
to Theorem 7.88. 


PROBLEM (7.7.21). Let Xy,X2,...,Xn be independent random vari- 
ables, each with characteristic function ¢(t). Obtain the characteristic 
function of 


Yn = Gn + bp(X1 + Xo +--+ + Xn), 


where a, and b, are arbitrary real numbers. 

Suppose that (t) = elt" where 0 < a < 2. Determine a, and by 
such that Y;,, has the same distribution as X, for n = 1,2,.... Find the 
probability density functions of X; when a = 1 and when a = 2. (Oxford 
1980F) 


SOLUTION. By Theorem 7.87, the characteristic function of Y, = ay, + 
bn (Xy + X2+---+ Xp) is given by 


dy, (t) = e"" (Plbnt))”, 
where ¢ is the common characteristic function of the X;. 
If o(t) = el", where 0 < a < 2, then the characteristic function of Y;, 
becomes 


dy,, (t) = citan (e7lntl® ‘ea 
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Now, Y, has the same distribution as X, for n = 1,2,... if and only if they 
have the same characteristic function, i.e., 


eitan (e-lbntl* yn _ e-lel® for allt ER 
iii citan o—nlbnt|* _ —l|t|* for alte R 
diol citan e—nlbnt|*+lt|* _ 7 for alte R 
peas eitant(1—nlbn|*)|t1* _ 4 for alte R 
= itdn + (1 — 1 |bn|*) |t* = 0 aa 


this is equivalent to (since the real part and the imaginary part in the above 
equation must equal to zeros) 


An = 0, 

(1-1 |bn|*) |él" = 0, 

An = 0, 

1—n|b,|* =0, since t is arbitrary 


ae = 
bab Sales 


an =O), 
Vb, =t(2)Ve, 


n 





If a = 1, then the characteristic function of X; becomes 


g(t) =e", 
which we recognize by Example 7.80 as the characteristic function of the 


Cauchy distribution. We deduce that Xj, has this distribution by appealing 
to Theorem 7.88. Thus, the probability density functions of X1 is 


1 
fx, (2) = m1 +22) 


If a = 2, then the characteristic function of X; becomes 


for x ER. 


o(t) =e, 
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which we recognize by Example 7.82 as the characteristic function of the 
normal distribution with mean pp = 0 and variance g? = 2. We deduce 
that X, has this distribution by appealing to Theorem 7.88. Thus, the 
probability density function of Xj, is 


fx, for zvER. 


1 1 
(x) = ee exp(—72") 


CHAPTER 8 
The main limit theorems 


8.1. The law of averages 


EXERCISE (8.8). Let Z, be a discrete random variable with mass func- 
tion 


1 1 
P(Z, =n“) = -, P(Z, =0) =1--. ’ 
n n 


Show that Z, converges to 0 in mean square if and only if a < 5. 


SOLUTION. Z, converges to 0 in mean square as n — oo if and only if 
E(Z2) + 0 as n— oo. But 














B(Z2) = (1 — =) + (n®(1 = =). 
1 


Seen 























Thus E(Z?) > 0 as n — oo if and only if = + O0asn- oo. This is 
equivalent to 1 — 2a > 0, giving that a < 5 





EXERCISE (8.9). Let Z1, Zz... be a sequence of random variables which 
converges to the random variable Z in mean square. Show that aZ, +b —- 
aZ +b in mean square as n — oo, for any real numbers a and b. 


SOLUTION. For any real numbers a and 8, 


























E([a(2n — 2)]”) 


= @E([zn — z]*) > 0 as Nn — ©, 


E([(aZn + b) — (aZ + 6)]?) 























since E([z, — z]”?) > 0 as n > 0. Hence, aZ, +b 4 aZ +b in mean square 
as mn —> oo, for any real numbers a and b. 
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EXERCISE (8.10). Let N,, be the number of occurrences of 5 or 6 in n 


throws of a fair die. Use Theorem 8.6 to show that, as n > co, 


1 
—Nn- = in mean square. 


3 
SOLUTION. We have that 
2 1 
P(N, =k) = i (<)r-F( =) for k = 1,2,...,n. 
The binomial theorem states that 


(a+a)"= se .) ata 


k=0 


Take derivative of this differentiate this equation with respect to x to obtain 


n(a +2)” t= 4(p)e ae (1) 


Multiply both sides of this equation by x to obtain 


n(at2)*le = > (2) ae 


k=1 


Take derivative of this equation with respect to x to obtain 


: 2 (7 \gn-kgk-] = n(a+2)*14+n(n—1)e(at+a)"*. (2) 
(i) 


8.1. THE LAW OF AVERAGES 



























































1 1 
B(~ Nn) = a ty Nn) 
a Ae 1 
eae k “\n—-k k 
=e) GIG) 
k=0 
1, fn\,2 i 
a4 Se k n—-k k-1 
3n ()@) (3) 
k=0 
ee ee ae ee 
= 3n(3 + 3) 
1 
=a by (1), 
and 
1 72 Le (HV 2 ce 
E( a Nn) = 3 k i (3) (3) 
k=0 
1 n\ 2 1 
eae, ke? “\n—ky = \k-1 
sa GGG 
k=0 
n+n(n—2)% 
= by (2). 
373 y (2) 
Then 
1 1 1 2 
B(—Nn ~ 5)? = E(GN2- Nn +5) 
1 yaa | 
= E(N3) - 5E(-Nq) +5 
n+n(n—1)% ee 
= 32 9+ 9 >0 as 1 —> Oo. 
Hence, as n > 00, 
1 ‘ 
—Nn- = in mean square. 
n 3 
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EXERCISE. Show that the conclusion of the mean-square law of large 
numbers, Theorem 8.6, remains valid if the assumption that the X; are in- 
dependent is replaced by the weaker assumption that they are uncorrelated. 


SOLUTION. This is a straightforward calculation.We write 


Sy, = Xy+Xot+---+ Xp 


8.2. Chebyshev’s inequality and the weak law 


EXERCISE (8.20). Prove the following alternative form of Chebyshev’s 
inequality: if X is a random variable with finite variance and a > 0, then 


P(|X —E(X)| >a) < =yvar(X). 


SOLUTION (1). By Markov’s inequality, Theorem 7.63, applied to the 
positive random variable (X — E(X))?, 


P(X — 


























E((X —E(X))’) _ 1 









































E(X)| > a) = P((X — E(X))? > a) 


IA 
l 
s 

pezs 

cs) 


a 


as required. 


SOLUTION (2). We have that 




















PUX-BOO|>a)= fo dFx(a) 

[e -E(X)P x) since if ja —E(a Ai on BIE? 
ears a2 dF'x (x) f |x —E(x)| >a, th a asi 
<5 f te- B00 arte) 
= Svar(X) recall that var(X) = i [x — E(X)|2 dFx (2). 


This method can also be applied to prove Markov’s inequality. Recall 
Markov’s inequality, Theorem 7.63: For any non-negative random variable 


x, 











ies 
——~ 
ars 








P(X >t)< for t > 0. 


Indeed, for t > 0, 


8.2. CHEBYSHEV’S INEQUALITY AND THE WEAK LAW 


RX ah) = dF'x (x) 


x>t 


i. ees 
x>t t 


IA 


IA 











E(X) 





ee recall that 


t 


1 (oe) 
al xdF x(x) since X > 0 
0 














E(X) 


x 
since if x >t, then " > 1 


[O sare(o 
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EXERCISE (8.21). Use Chebyshev’s inequality to show that the proba- 
bility that in n throws of a fair die the number of sixes lies between 4n—./n 


and in + v/n is at least 3. 


6 


SOLUTION. Let X, be the number of sixes in n throws of a fair die. 


Then 


6 
if and only if 





X,— 5] < vi. 


1 1 


We apply Chebyshev’s inequality to the random variable Y = X;, — &, to 


find that 


Hence 














The binomial theorem states that 














for k = 0,1,2,...,n. 


2) = AE) 


yf. 
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tay <5 (*)artet 


k=0 


Take derivative of this equation in x to obtain 


n(a+x)” ada ( para! 


Multiply both sides of this equation by x to obtain 


n(a +a)" t= Dow (perts (1) 


We apply this with a = - and «2 = é to obtain 














Take derivative of (1) in x to obtain 


niata)” 1 +4n(n—-1)(at+2)"?7 = 3 k? (;) 7 iam a 


k=1 


Multiply both sides of this equation by x to obtain 
n(a+a)” 2 +n(n—1)(a+a)" 22 2,2 “rel; Ja n— kpn—k 


We apply this with a = 2 and 2 = é to obtain 


E(Xz) 


“ af T\ One lig 
Learn 
n=0 

n  n(n—1) 

6° 36 

n* + 5n 

g60°* 
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Alternatively, we note that X has the binomial distribution with parameters 
n and p= é: hence E(X) = np = an and E(X?) = npq + n?p? = niin 


36 
(see Exersice 2.37). Thus 













































































E((Xy — 4n)?) = E(X2) — Lne(X,) + @ 
6 Wee We 3G 
_ nw +5n a 
36 18 36 
_ dn 
36’ 
giving that 
n 1 n 
P(|x - =| < -E((xX, — —)? 
(Xn 5 2G ES (Xn q)) 
sl 5n 
~ n 36 
8 
S86: 


Therefore 


rifts 3] <vmat ms —2] 2 va 


SS 
36 
31 
367 


as required. 


EXERCISE (8.22). Show that if Z, — Z in probability then, as n > co, 


aZpn +b>aZ+b in probability, 


for any real numbers a and 0. 


SOLUTION. If a = 0, then the result is trivial. If a 4 0, then, for e > 0 
be given, 





P(|(aZn + b) — (aZ + 6)| > €) = P(lal |(Zn — 4] > ©) 


= P(((Zn— Z| > 77) +0 as N — 00, 











since Z,, + Z in probability. Thus, as n > ov, 
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aZp,+b>aZ+b in probability, 


for any real numbers a and 6. 


8.3. The central limit theorem 


EXAMPLE (8.30). (Statistical sampling) The central limit theorem 
has many applications in statistics, and here is one such. An unknown 
fraction p of the population are jedi knights. It is desired to estimate 
p with error not exceeding 0.005 by asking a sample of individuals (it is 
assumed they answer truthfully). How large a sample is needed? 


SOLUTION. Suppose a sample of n individuals is chosen. Let X; be the 
indicator function of the event that the ith such person admits to being a 
jedi knight, and assume the X; are independent, Bernoulli random variables 
with parameter p. Write 


Then S$, has the binomial distribution with parameters n and p (see Exer- 
cise 3.31). 


EXERCISE (8.32). A fair die is thrown 12,000 times. Use the central 
limit theorem to find values of a and b such that 


b 
eee 
P(1900 < S' < 2200 ~ | —=e" 2” dr, 
( dae err 


where S is the total number of sixes thrown. 


SOLUTION. Let X; be the indicator function of the event that the ith 
throw is a six, and assume the X; are independent, Bernoulli random vari- 
ables with parameter p = 7 with mean and variance given by 
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I 

| 
— 
2 


Write 


The standardized version of S is 


S— np 
o/n 
S — 12,000 - 4 


\/12,000- 3 


Zn = 





By the central limit theorem, 
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1 





p 1900 = 12, 000 = 12,0005 _ 2200 — 12,000 §, 
P(1900 < S < 2200) = 
\/12,000- 3 a 000. & \/12,000- 3 
—100 — 12,000: 2 200 





— p 








< < 
,/12, 000 - ce 


= P(-vo < SMOOVE < av8, 


2v6 1 1,2 
~~ ——e 2” dz 
i. V2T 


Thus a = —\V/6 and b = 2V6. 














NI 


) 
1/12, 000 ae 
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EXERCISE (8.33). For n = 1,2,..., let X;, be a random variable having 
the gamma distribution with parameters n and 1. Show that the moment 
generating function of Z, = (Xn —n)/1/n is 





S t 
Mn(t) =e RC, = a i 
and deduce that, as n — oo, 
z 1 _1,2 
P(Z, <2) > e 2” du for x ER. 
oo V2T 


SOLUTION. For the first part, the density function of X,, is 


1 n-1,-2z if 0 
Tar ¢ ifx>0, 

xv 
Fxa( rf iee< 0: 


The moment generating function of X,, is 


T(n) 
=— i “ ‘ae as : du by the substitution u = (1 — t)a 
— T(n) Jo ‘1-8 1-t y = 
1 i n-1-—u 
= ue" du 
T(n)(1 — t)” Jo 
1 oo 
= ————_T'(n) since | gr le-* de = T(n) 
T(n)(1 — t)” 5 
1 
=a) ift<1. 


The moment generating function of Z, = (X», —1)/,/n is 
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t) by (7.51) 


as required. 
For the last part, we rewrite M(t) in the form 








-t 
€ 
M,(t) = ve 
Jn 
Taking logarithms, we have 
log My, (t) = /n(-t — Vnlog(1 — ae since loge * = —t 
Jn 
= J/n[-t — /n( i . a + o( : ))] use Taylor’s theorem to expand log(1 — as 
Vn 2n 3nf/n' <n y Jn 
Pt 48 8 aid 12 
am Saga ne Bs as n — 00, 
giving that 
142 
M,(t) > e2 as Nn — ©, 


and the result follows from Theorem 8.27. That is, as n > ov, 


P(Zn <x) > / e 3” du for ER. 
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8.4. Large deviations and Cramér’s theorem 
8.5. Convergence in distribution, and characteristic functions 
8.6. Problems 


PROBLEM (8.6.1). Let X 1, X2,... be independent random variables, 
each having the uniform distribution on the interval (0,a), and let Z, = 
max{X1, Xo,..., Xn}. Show that 

(a) Z, — a in probability as n — oo. 

(b) /Z, > Va in probability as n > oo. 

(c) if U, =n(1— Z,) and a = 1, then 


l—-e* if 
Fn <#)=4 e€ ifx>0, 


0 otherwise, 


so that U, converges in distribution to the exponential distribution as n > 
oo. 


SOLUTION. (a) The distribution function of Z, is 


Fy. (2) = P(A 2) 
= P(X; <2,i=1,2,...,n) 














= [[®e: <G) by independence 
i=l 
n 
= Il ye (x) 
i=l 
0 ifxa<0O, 0 ifa<0, 
=(2 if0<2<a, since Fx,(t)= 42 if0<a<a, 
1 ifa>a, 1 ifv>a. 
For € > 0, 
P(|Zn — al > €) =P(Z, —a>€ or Z, —a < —e) 
=P(Z, —a < —e) since the event Z, — a > € does not happen 
=P(Zn < a-—e) 








_ jo ifa—e<0, 
7 fo-" if0<a-—e<a. 
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Since =“ < 1 if0 < a—e <a, it follows that igo >O0asn> wif 
0<a-—e<a. Thus P(|Z, —a| > €«) ~ 0 asn—- oo. Hence Z, > a in 
probability as n > oo. 


(b) For € > 0, 








P(|VZ, — val > €) = POs ti Wa Se ern Za 4a <8) 
=P(VZn — Va < —€) 
( 





iS, 











=P(/Zn < Va-6) 
a ko if /a—e <0, 
Pe GO /a Sey). OS 4G eS 47a, 


oO 


7 if /a—e <0, 
[eee OR Wa see Wa. 


Since 0 < a—e€ < Va, so 0 < (\/a— ©)” < a, implying 0 < et ean 
if0 < /a-—e < Va. It follows that (ve-)7l" _, Q as n + 00 if 0 < 


Ja—€< Ja. Thus P(|\VZ, — Va] > €) + 0asn— oo. Hence /Z, > Ja 
in probability as n > oo. 
(c) We have that 





P(U, < x) = P(n(1 — Z,) < 2) 

















1 if 2 <0; 
if0< *=*<1, 
0 ip ed, 


If  < 0, then =* > 1, so PU, < 0) = 0. If e > 0 and n > a, then 
0<"=*<1,s0 
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P(Un < ©) =1-(——) 
=1-(1 “yn >l-—e? as 2 —> OO. 
n 


Thus 


l-e* if 
Fun <#)={ e ie SO; 


0 otherwise, 


so that U, converges in distribution to the exponential distribution as n > 
oo. 


PROBLEM (8.6.2). By applying the central limit theorem to a sequence 
of random variables with the Bernoulli distribution, or otherwise, prove the 
following result in analysis. If0 <p=1-—q<1landz > 0, then 





n k n-k gee! —1,? 
Ss" pq +2 e 2% du as 2 — 00, 
k 0 20 
where the summation is over all values of & satisfying np — x,/npq < k < 
np + x./npq 


SOLUTION. Let X1, X2,... be independent, Bernoulli random variables 


with parameter p, each with mean jz = p and non-zero variance a? = p(1—p) 
. Write 
n 
SSX, 
i=l 
and so 


_ Sy- np 

= “aya. 
Sy — np 
np(1 =p) 


Zn 


By the central limit theorem, Theorem 8.25, 
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x 
if 
P(Zn <x) > . ae du forz ER. 
a 
Thus 
P(lZnl <2) > [ ae a ee 
n| S2& € U as N — 00, 
_¢ V2 
and so 





x 
P(|Z,| <2) > 2 | e3" du as N —> 00, (1) 
0 


since f° A ¢-2" du =2 te _¢-3" du. On the other hand 











np(1 — p) 
= P(np — x./npg < Sp < npt+ x,/npq). (2) 


The moment generating function of X; is 























Mx,(t) = E(e**) 
=e (1—p)+e"lp 
=pe'+1—p, 


giving that S, has moment generating function 


Let Y be the random variable that has the binomial distribution with pa- 
rameters p and n. Then Y has moment generating function 
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II 

ry 
xe 
> 


a nok 


(0 
=> (g) eek - ay 
" 


oa 


pe + p)” by the binomial formula 


Thus Mg, (t) = My(t 
ating function 


By the uniqueness theorem for the moment gener- 


Fs, (x) = Fy(2), 


where F's, (x) and Fy (x) are the probability distribution function of S,, and 
Y, respectively. Hence 


P(np — 2,/npg < Sn < np + x,/npq) = se — x,/npq < Y < np+2,/npq) 


where the summation is over all values of k bine np — x,/npq<k< 
np +2,/npq. From (1), (2), (3) we obtain 


x 
1 
ale har t2 | e723” du as n > oO, 
k 0 20 
where the summation is over all values of & satisfying np — x,/npq < k < 
np + £,/npq. 


PROBLEM (8.6.3). Let X, be a discrete random variable with the bi- 
nomial distribution, parameters n and p. Show that n~!X,, converges to p 
in probability as n — oo. 





SOLUTION. Since X, has the binomial distribution with parameters n 
and p, it follows that 


Mx, (t) = (pe' +1—p)”, 


and so 


8.6. PROBLEMS 349 





E(Xn) = M,,(0) 
10, 
E(X2) = M7(0) 
= np(1— p) + n?p?. 

















Since 








E([n~'X,, — pl’) = E(n-* X47 — 2pn! X,, + p*) 


















































= SE(3) — PE(X,) +9? 
= - np(1 — p) + n*p7] Pnp + p 
me P) Lp? — 2p? + p? 
_ pl~p) 
n 
=< 40 as n— oo since p(l—p) < 


n—!X,, converges to p in mean square as n — oo. By Theorem 8.14, n7!X), 
converges to p in probability as n > oo. 


PROBLEM (8.6.5). By applying the central limit theorem to a sequence 
of random variables with the Poisson distribution, or otherwise, prove that 





2 n 
—n | ut | | a 1 
e Cas ae ar) 5 as NM — OO. 
SOLUTION. Let X,, Xo,..., Xn be independent random variables, each 


having the Poisson distribution with parameter 1 with probability generat- 
ing function 














Gx,(s) = E(s**) = S> =. 


Let S, = X1+Xo4+--:Xn. Then S;,, has the probability generating function 


ALR 
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Ge (3) = II Gx, (s) by independence 


so that S,, has the Poisson distribution with parameter n. Hence mean and 
variance of S,, are given by E(S,,) = 0 and var(S;,,) = 1. Now, 

















giving by the central limit theorem, Theorem 8.25, that the limiting distri- 
bution of Z,, is the normal distribution with mean 0 and variance 1, giving 
that 


“0 
1 1 
=P(Z,<0)73 ——e 2” dr=- as mn > Oo 
(Zn $0) L= 2 





e "(l+n+ } +++ + —) as N — OO. 


PROBLEM (8.6.7). Use the Cauchy—Schwarz inequality to prove that if 
Xn, — X in mean square and Y, — Y in mean square, then X, + Y, > 
X +Y in mean square. 


SOLUTION. We have that 


E([(Xn + Yn) — (X + ¥)I?) 

= E([(Xn — X) + (Yn - YY?) 
= E((Xn — X)?) + E((¥n — Y)’) 
+ 2E((Xn — X)(¥n — Y)) 
E((Xn — X)?) + E((Yn — Y)?) 









































IA 





+ 2, /E((Xn — X)2)E((Yn — Y)?) by the Cauchy—Schwarz inequality, Theorem 7.30. 


But X, — X in mean square and Y, — Y in mean square, i.e, 


8.6. PROBLE 





E([X, — X]?) > 0 
E([¥, -Y]7) 0 

















MS 
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as n — CO 


as 2 —> OO, 














and so E((X;,— X)*) +2,/E((Xn — X)?) 


























E((Y¥n — Y)?) + 























E((Yn -—Y)?) 30 





as n —> oo. Thereofore, E([(Xn + Yn) — (X + Y)]?) — 0 as n > 00, ice, 





Xn+Yn—-> X +Y in mean square. 


PROBLEM 4. Use the Cauchy—Schwarz inequality to prove that if X, > 
Give an example of a sequence 




















X in mean square, then E(X,) > E(X). 








X 1, X9,... such that X, — X in probability but 





to E(X). 











SOLUTION. X,, — X in mean square, 











i Xt XI) 0 

















i.e, 


E(X,,) does not converge 


as 2 — OO 


For every n € N, let Y,, be the random variable that P(Y, = 1), then 







































































E((Xn — X)¥n) < VE((Xn — X)?)E(Y,?) 


= JE((X, — X)?) +0 

















E(Y,) = 1, E(Y,) = 1. Applying the Cauchy—Schwarz inequality to obtain 





But E((X,—X)Yp) = E(X,—X) = E(X,)—E(X). So E(X,) -E(X) > 0 
































as n + oo. Thus E(X,,) — E(X). 

















Example 8.19 is an example of a sequence X1, Xo,.. 






































in probability but E(X,,) does not converge to E(X). 

















. such that X, > X 


PROBLEM 5 (8.6.11). Adapt the proof of Chebyshev’s inequality to 


show that, if X is a random variable and 


a > 0, then 





P([X] 2 a) < —TE(g(X)), 


1 
7 g(a) 


for any function g: R — R which satisfies 
(a) g(x) = g(—2) for x ER, 
(b) g(x) > 0 for « £0, 


(c) g is increasing on [0, 00). 











352 8. THE MAIN LIMIT THEOREMS 


SOLUTION. If |X| > a> 0, then |X| 4 0 and so g(|X]|) > 0 by condition 
(b). Since g is increasing on [0,co) and by applying Markov’s inequality, 
Theorem 7.63, we have 














Ls 
P([X] 2 a) = P(g(|X|) 2 g(@)) < re E(g(|X|)). (1) 
If X > 0, then |X| = X, so g(|X|) = g(X). If X < 0, then |X| = —X, 
so g(|X|) = g(—X) = g(X) by condition (a). Therefore, in all cases we 


obtain 


g(|X|) = g(X) (2) 
Combine (1) and (2) to obtain 





P([X| >a) < a E(q(X)), 











as required. 


PROBLEM (8.6.14). Let (X,: n > 1) be a sequence of random variables 
which converges in mean square. Show that E([X;—Xm])? > 0 as m,n > 
Oo. 

If E(X,) = wand var(X;,) = o? for all n, show that the correlation between 
Xn and Xm converges to 1 as m,n — co. 


























SOLUTION. For the fist part, suppose X, — X in mean square as 
n — oo. We have that 


(Xn = ay = [(Xn _ xX) Te (x a Rall 
SO HO = eX 
































IA 
ct 
pe 
= 
a 
3 
S) 
No 
< 


((Xm — X)?) 
+ 2./E((Xn — X)?2)E((X — Xm)?) by the Cauchy—Schwarz inequality. 














Thus 








0 < E((Xn — Xm)”) S$ E((Xn — X)*) + 2VE((Xn — X)?)E((X — Xm)?) + E((Xm— X)*) (1) 
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Since X, — X in mean square as n — oo, so E((X, — X)?) — 0 and 
E((Xm — X)”) as n + oo. Therefore 























E((Xn — X)?) + 2,/E((Xn — X)2)E((X — Xm)?) + E((Xm — X)?) +0 asm,n— oo. (2) 














Combine (1) and (2) to obtain E([Xn — Xm])? + 0 as m,n — oo. 
For the second part, recall that the correlation of the random variables 
Xp and Xm is 








E((Xn = Lt) (Xm = LL) 
/var(X;,)var(Xm) 
E((Xn _ Lt) (Xm a LL) 
o2 
E(XnXm— (Xn + Xm) + v7) 
o2 
E(XnXm) — U(E(Xn) + E(Xm)) + ye 





(Xn, Xm) = 










































































2 
== since E(X,) = E(Xm) = ps 


Thus 

















p(Xn, Xm) = 
We also have 
0 < |E(X,Xm) — E(X2)| = |E(Xn(Xm — Xn)) 


< JE(X2)E((Xm — Xn)?) by the Cauchy—Schwarz inequality 
= Vit + oF VE(Xn — Xn?) 70 

















since E(X?2) = pi? + 0? and by the first part. Therefore 











lim E(X;Xm) = lim E(X?) = p? +0? (2) 


n,mM—>oo n—-oco 












































since 0? = var(Xn) = E(X2) — p?, so E(X2) = p? +0? for all n. 
Combine (1) and (2), we have 
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lim p(Xn,Xm) = lim 


m,n—oo m,n—-oo o2 
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